Artificial Intelligence Full Course 2026 | Artificial Intelligence Tutorial | AI Course |Simplilearn

Key Takeaways

[Music] Hey there, welcome to our artificial intelligence full course by simply learn. Have you ever wondered how machines can talk like humans, write stories, design art, or even build apps? It almost feels like something straight of a sci-fi movie. But with AI, it's happening right now. From healthcare to finance, creative industries, and tech, AI is transforming the way the world works. And in 2025, learning AI isn't just optional. It's one of the most powerful skills you can have to futureproof your career. That's exactly why we have designed this course to be fun, beginner friendly, and hands-on. No boring theory dumps here. You'll actually build exciting real world projects while learning with concepts step by step. We'll start with an introduction to AI and map out AI road map so you know exactly where you're headed. Then we'll dive into powerful topics like large language models, transformers, deep learning, reinforcement learning, recorded neural networks, and even multimodel AI. Don't worry, we'll keep it simple, practical, and easy to follow. And it doesn't stop at theory. You'll get to see AI in action. We walk you through creating your first AI powered website, building iOS app with AI, running a full demo of LLM chatbots, and yes, even creating your very first AI clone. And along the way, we'll also tackle real world challenges like AI bias, work on exciting AI projects, plus we will prepare you with machine learning interview questions, scraper tools that will give you an extra edge. And by the end of this course, you won't just understand AI, but you'll be using it confidently in building machine learning models and also that can shape your career. So if you're ready to dive into the world of AI and be a part of the revolution with Simplearn, let's get started now. Now before we move on, here's a quick information. If you are interested in growing your career in AI and machine learning, this course is a great way to start. The professional certificate in AI and machine learning by simply learn port University online and IBM will help you master all the key AI skills like chat GPD lms deep learning and agentic AI through live classes hands-on project and in just 6 months you'll work on real world industry projects use 18 plus tools like Python, TensorFlow and also earn certificate from Perdu and IBM. You'll also get career support including resume help, mock interviews and job assistant by simply. So what are you waiting for? Hurry up and enroll now and you can find the course link below. >> Introduction of artificial intelligence and machine learning. By the end of this lesson, you will be able to define artificial intelligence. Describe the relationship between artificial intelligence and data science. Define machine learning. Describe the relationship between machine learning, artificial intelligence, and data science. describe different machine learning approaches. Identify the applications of machine learning. Let's understand how the field of artificial intelligence emerged. Let's first understand the reason behind the emergence of AI. Data economy is one of the factors behind the emergence of AI. It refers to how much data has grown over the past few years and how much more it can grow in the coming years. When you look at this graph, you can clearly understand how the volume of data has grown. You can see that since 2009, the data volume has increased by 44 times with the help of social websites. The explosion of data has given rise to a new economy and there is a constant battle for ownership of data between companies to derive benefits from it. Now that you know that data has grown at a rapid pace in the past few years and is going to continue to grow, let's understand the need for AI. As you know, the increase in data volume has given rise to big data which helps manage huge amounts of data. Data science helps analyze that data. So the science associated with data is going toward a new paradigm where one can teach machines to learn from data and drive a variety of useful insights giving rise to artificial intelligence. Now you may ask what is artificial intelligence? Artificial intelligence refers to the intelligence displayed by machines that simulates human and animal intelligence. It involves intelligence agents, the autonomous entities that perceive their environment and take actions that maximize their chances of success at a given goal. Artificial intelligence is a technique that enables computers to mimic human intelligence using logic. It is a program that can sense, reason, and act. Let's look at some of the areas where artificial intelligence is used. Artificial intelligence is redefining industries by providing greater personalization to users and automating processes. One example of artificial intelligence in practice is self-driving cars. Self-driving cars are computercontrolled cars that drive themselves. In these cars, human drivers are never required to take control to safely operate the vehicle. These cars are also known as autonomous or driverless cars. Let's see how Apple uses AI. iPhone users can experience the power of Siri, the voice. It simplifies navigating through your iPhone as it listens to your voice commands to perform tasks. For instance, you can ask Siri to call your friend or to play music. Siri is fun and is extremely convenient to use. Another example is Google's Alph Go, which is a computer program that plays the board game Go. It is the first computer program to defeat a world champion at the ancient Chinese game of Go. Amazon Echo is another product. It's a home control chatbot device that responds to humans according to what they are saying. It responds by playing music, movies, and more. If you've got compatible smartome devices, you can tell Echo to dim the lights or turn appliances on or off. You can use AI and chess. And here is an example of a concierge robot from IBM called IBM Watson. The IBM Watson AI has typically been in the headlines for composing music, playing chess, and even cooking food. Let's move ahead and look at some sci-fi movies with a concept of artificial intelligence. The films featuring AI reflect the everchanging spectrum of our emotions regarding the machines we have created. Humans are fascinated by the concept of artificial intelligence and this is reflected in the wide range of movies on AI. Recommendations systems are used by a lot of e-commerce companies. Let's see how they work. Amazon collects data from users and recommends the best product according to the user's buying or shopping pattern. For example, when you search for a specific product in the Amazon store and add it to your cart, Amazon recommends some relevant products based on your past shopping and searching pattern. So before you buy the selected product, you get recommendations based on your interest. And there is a possibility that you may also buy the relevant product with the selected product. If not, you have the chance to compare the selected product with the recommended products. Now let's move ahead and understand the relationship between artificial intelligence, machine learning, and data science. Even though the terms artificial intelligence, AI, machine learning, and data science fall in the same domain and are connected to each other, they have their specific applications and meaning. Let's try to understand a little about each of these terms. Artificial intelligence systems mimic or replicate human intelligence. Machine learning provides systems the ability to automatically learn and improve from the experiences without being explicitly programmed. Data science is an umbrella term that encompasses data analytics, data mining, machine learning, artificial intelligence, and several other related disciplines. Let's look at the flow diagram and try to understand the relationship between AI, machine learning, and data science. Interestingly, ML is also an element of artificial intelligence. So the first step is data gathering and data transformation. This step basically comes under data science. Data transformation is the process of converting data from one format or structure into another format or structure. Data transformation is important to activities such as data management and data integration. After gathering data, we would want to use the data to make predictions and derive insights. In order to get predictions out of the data set, we use machine learning techniques such as supervised learning or unsupervised learning. On an overview level, supervised and unsupervised learning are the machine learning techniques used to extract predictions from a given data set. >> Now, you must be thinking where deep learning comes into the picture. Deep learning is a sub field of machine learning involved with algorithms. It uses artificial neural networks which are modeled on the structure and performance of neurons in the human brain. Deep learning is most effective when there isn't a clear structure to the data that you can just exploit and build features around. >> Now the next step in the flow diagram is to get insights from predictions being made. In order to do so, you need to use data analysis, which actually is the process under data science. Now, when you are done with all of these, you must want your data to perform some actions. This is where AI comes into the picture. Artificial intelligence combines predictions and insights to perform actions based on the human decision and automated decision. Now let's move ahead and understand the relationship between artificial intelligence, machine learning and data science. Let's look at the relationship between artificial intelligence and machine learning. Artificial intelligence is the engineering of making intelligent machines and programs. Machine learning provides systems the ability to learn from past experiences without being explicitly programmed. Machine learning allows machines to gain intelligence thereby enabling artificial intelligence. Let's now understand the relationship between machine learning and data science. Data science and machine learning go hand in hand. Data science helps evaluate data for machine learning algorithms. Data science covers the whole spectrum of data processing while machine learning has the algorithmic or statistical aspects. Data science is the use of statistical methods to find patterns in the data. Statistical machine learning uses the same techniques as data science. Data science includes various techniques like statistical modeling, visualization, and pattern recognition. Machine learning focuses on developing algorithms from the data provided by making predictions. So what is machine learning? Machine learning is the capability of an artificial intelligence system to learn by extracting patterns from data. It usually delivers quicker, more accurate results to help you spot profitable opportunities or dangerous risks. Now you must be curious to understand the features of machine learning. Machine learning uses the data to detect patterns in a data set and adjust program actions accordingly. Pattern detection can be defined as the classification of data based on knowledge already gained or on statistical information extracted from the patterns. It focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data by using a method called reinforcement learning. It uses external feedback to teach the system to change its internal workings in order to guess better next time. It enables computers to find hidden insights using iterative algorithms without being explicitly programmed. Machine learning uses algorithms that learn from previous data to help produce reliable and repeatable decisions. It automates analytical model building using the statistical and machine learning algorithms that tease patterns and relationships from data and express them as mathematical equations. Let's understand the different machine learning approaches. So what is the actual difference between traditional programming and machine learning? In traditional programming, data and program is provided to the computer. It processes them and gives the output. However, the machine learning approach is very different. In machine learning, algorithms are applied on the given data and output. The result of the applied algorithm and calculations is a learning model that helps machine to learn from the data. In traditional programming, you code the behavior of the program. But in machine learning, you leave a lot of that to the machine to learn from data. Now let's first understand the traditional programming approach. Traditionally, you would hardcode the decision rules for a problem at hand, evaluate the results of the program, and if the results were satisfactory, the program would be deployed in production. If the results were not as expected, one would review the errors, change the program, and evaluate again. This iterative process continues till one gets the expected result. What is the machine learning approach? In the new machine learning approach, the decision rules are not hardcoded. The problem is solved by training a model with the training data in order to derive or learn an algorithm that best represents the relationship between the input and the output. This trained model is then evaluated against test data. If the results were satisfactory, the model would be deployed in production. And if the results are not satisfactory, the training is repeated with some changes. Machine learning techniques. Machine learning uses a number of theories and techniques from data science. Here are some machine learning techniques. Classification, categorization, clustering, trend analysis, anomaly detection, visualization, and decision making. Let's look at these techniques. Classification is a technique in which the computer program learns from the data input given to it and then uses this learning to classify new observations. Classification is used for predicting discrete responses. Classification is used when we are training a model to predict qualitative targets. Categorization is a technique of organizing data into categories for its most effective and efficient use. It makes free text searches faster and provides a better user experience. Clustering is a technique of grouping a set of objects in such a way that objects in the same group are most similar to each other than to those in other groups. It is basically a collection of objects on the basis of similarity and dissimilarity between them. Trend analysis is a technique aimed at projecting both current and future movement of events through the use of time series data analysis. It represents variations of low frequency in a time series. The high and medium frequency fluctuations being out. Anomaly detection is a technique to identify cases that are unusual within data that is seemingly homogeneous. Anomaly detection can be a key for solving intrusions by indicating a presence of intended or unintended induced attacks, defects, faults, and so on. Visualization is a technique to present data in a pictorial or graphical format. It enables decision makers to see analytics presented visually. When data is shown in the form of pictures, it becomes easy for users to understand it. Decision making is a technique or skill that provides you with the ability to influence managerial decisions with data as evidence for those possibilities. Now, I am sure you have a better understanding of the overview of machine learning. So, let's look at some realtime applications of machine learning. Artificial intelligence and machine learning are being increasingly used in various functions such as image processing, robotics, data mining, video games, text analysis, and healthcare. Let's look at each of them in more details. So what is image processing? It is a technique to convert an image into a digital format and perform some operations on it so as to induce an enhanced image or to extract some helpful information from it. Let's look at some of the examples of image processing. Facebook does automatic face tagging by recognizing a face from a previous user's tagged photos. Another example is optional character recognition which scans printed docs to digitize the text. Self-driving cars are another big example of image processing. Autopilot is an optional drive system for Tesla cars. When autopilot is engaged, cars can self- steer, adjust speed, detect nearby obstacles, apply the brakes, and park. Now, let's see how robotics uses machine learning. Robots are machines that can be used to do certain jobs. Some of the examples of robotics are where a humanoid robot can read the emotions of human beings or an industrial robot is used for assembling and manufacturing products. So let's look at some realtime applications of machine learning. Let's see what data mining is. It is the method of analyzing hidden patterns in data. Let's look at some of the applications of data mining. It is used for anomaly detection to detect credit card fraud and to determine which transactions vary from usual purchasing patterns. It is also used in market basket analysis which is used to detect which items are often bought together. It can be used for grouping where it classifies users based on their profiles. Machine learning is also applied in many video games in order to give predictions based on data. In a Pokemon Go battle, there is a lot of data to take into account to correctly predict the winner of a battle. And this is where machine learning becomes useful. A machine learning classifier will predict the result of the match based on this data. Let's move on to one of the most popular applications of machine learning, which is text analysis. It is the automated process of obtaining information from text. One example of text analysis is spam filtering which is used to detect spam in emails. Another example is sentimental analysis which is used for classifying an opinion as positive, negative or neutral. It detects public sentiment in Twitter feed or filters customer complaints. It is also used for information extraction such as extracting specific data address, keyword or entities. There are many applications of machine learning in the healthcare industry identifying disease and diagnosis, drug discovery and manufacturing, medical imaging diagnosis and so on. Some of the companies that use machine learning have revolutionized the healthcare industry are Google Deep Mind Health, Biobeats, Health Fidelity, and Ginger.io. >> Today, we'll take you through the exciting road map of becoming an AI engineer. If our content psiques your interest and helps fuel your curiosity, don't forget to subscribe to our channel. Hit that bell icon so you never miss an update. Now let's embark on this AI journey together. As artificial intelligence continues to revolutionalize various industries, AI engineers stand at the forefront of this technological wave. These professionals are essential in crafting intelligence systems that address complex business challenges. AI projects often stumble due to poor planning, subpar architecture or scalability issues. AI engineers play a crucial role in overcoming these hurdles by merging cuttingedge AI technologies with strategic business insights. So in this video, we'll guide you through the essentials of becoming an AI engineer. Let's start with the basics. What does an AI engineer do? An AI engineer builds AI models using machine learning algorithms and deep learning neural networks. These models are pivotal in generating business insights that influence organizational decision making. From developing applications that leverage sentiment analysis for contextual advertising to creating systems for visual recognition and language translation, the scope of an AI engineer's work is vast and impactful. So to succeed as an AI engineer, you need a blend of technical progress and soft skills. So now let's break down this 8th month plan. Month one, computer science fundamentals and beginner's Python. So before we delve into AI, it's crucial to establish a strong foundation in computer science. This month you should focus on the following topics. Data representation. Understanding bits and bytes, how text and numbers are stored and the binary number system is foundational for everything in computing. This knowledge helps in comprehending how computers interpret and process data. Now next comes computer networks. Learn the basics of computer networks including IP addresses and internet routing protocols. It's essential to understand how data travels across networks using UDP, TCP and HTTP which form the backbone of the internet and the worldwide web. Next comes programming basics. Begin with the basics of programming like variables, strings, numbers, conditionals, loops, and algorithm basics. These fundamentals will allow you to write and understand simple programs. Simultaneously, you'll also start with Python, the preferred language for AI. So learn about variables, numbers, strings, lists, dictionaries, sets, pupils, and control structures like if conditionals and for loops. And then move on to functions and modules. Understand how to create functions including lambda functions and work with modules by using pip install to add functionality to your projects. Next comes file handling and exceptions. You should also practice reading from and writing to files as well as handling exceptions to make your programs more robust. finally grabs the basics of classes and objects which are crucial for writing organized and efficient code. So this comprehensive overview sets the stage for more complex programming tasks that you'll encounter in the following months. Now in month two, you'll move on to data structure algorithms and advanced Python. So building on the foundations from month one, we'll now delve into data structure and algorithm. So familiarize yourself with the concept of bigger notation to understand the efficiency of different algorithms and data structures. Learn about edits, linking list, hash tables, tags, cues, trees, and graphs. Mastering these structures will allow you to store and manipulate data effectively. Now, next comes algorithms. You should explore algorithms such as binary search, bubble sort, quick sort, merge sort, and recursion. These are essential for optimizing your code and parallelly you'll advance your Python skills. So, you can dive into inheritance, generators, iterations, list comprehensions, decorators, multi-threading, and multipprocessing. These topics will enable you to write more efficient and scalable code. So this month's learning prepares you to handle complex data operations and enhance your coding efficiency. Now in month three, you'll move on to version control, SQL and data manipulation. So in the third month, the focus shifts to collaboration and data management. Number one, version control. So understand the importance of version control systems, especially Git and GitHub. So learn basic commands such as add, commit, and push. You should also learn how to handle branches, reward changes and understand concepts like head diff and merge. So these skills are invaluable for tracking changes and collaborating with other developers. Next, pull requests. Master the art of creating and managing pull requests to contribute to collaborative projects. Next, we'll dive into SQL for managing databases. So first we'll start with SQL basics. So learn about relational databases and how to perform basic queries and then you'll move on to advanced queries. Understand complex query techniques such as CT, subqueries and window functions. And then comes joins and database management. So study different types of joins like left, right, inner and full joint. You should also learn how to create databases, manage indexes and write store procedures. Additionally, you'll use numpy and pandas for data manipulation and learn basic data visualization techniques. This comprehensive skill set will be crucial as you move into more advanced data science topics. So now in month four you'll deal with maths and statistics for AI. So mathematics and statistics are the backbone of AI and this month is dedicated to these critical subjects. So first learn about descriptive versus inferial statistics, continuous versus discrete data, nominal versus ordinal data, measures of central tendency like mean, median, mode and measures of dispersion like variance and standard deviation. After that understand the basis of probability and delve into normal distribution, correlation and coariance. After which you should move on to advanced concepts. So you can study the central limit theorem, hypothesis testing, p values, confidence intervals and so on. In parallel, you should also study linear algebra and calculus. So in linear algebra, learn about vectors, matrices, ean values and egan vectors. And in calculus, cover the basics of integral and differential calculus. So this mathematical foundation is essential for developing and understanding AI models, setting you up for success as you transition into machine learning. Now in month five comes explanatory data analysis which is EDA and machine learning. So with a solid foundation in math and statistics, you are now ready to delve into machine learning. Number one, pre-processing. Learn how to handle NA values, treat out layers, perform data normalization, and conduct feature engineering. You should also understand encoding techniques such as one hot and label encoding. We'll also explore supervised and unsupervised learning with a focus on regression and classification and learn about linear models like linear and logistical regression and nonlinear models like decision tree random forest etc. And then understand how to evaluate models using metrics such as mean squared error mean absolute error mean for regression and accuracy precision recall etc. Then comes hyperparameter tuning. Learn about techniques like grid search CV and random search CV for optimizing your models after which you'll move to unsupervised learning. Here you can study clustering techniques like K means and hierarchical clustering and delve into dimensionality reduction with PCA. So this month's focus on EDA and model building will prepare you for more complex AI applications. Transitioning to the next phase, you'll begin to work on deploying these models and real world scenarios. So in month six comes MLOps and machine learning projects. So this month we'll cover the operational aspects of machine learning and work on practical projects. So in MLOps basics, learn about APIs particularly using fast API for Python and server development. Understand DevOps fundamentals including CI/CD pipelines and containerization with Docker and Kubernetes. You should also gain familiarity with at least one cloud platform like AWS or Azure. Now in month seven comes deep learning. So in this month we delve into the world of deep learning. So number one comes neural network. So learn about neural networks including forward and backward propagation and build multi-layer perceptrons after which you'll move on to advanced architectures. So here explore convolutional neural networks which are CNN's for image data and sequence models like RNNs and LSDNs. So this deep learning knowledge will be crucial as you move into specialized areas of AI in the final month. Now in the final month, the eighth month comes NLP or computer vision. So the final month you have the option to specialize in either natural language processing, NLP or computer vision. So first we'll deal with NLP track. So here you should learn about rejects text representation methods like count vectorizer TF b word to vec embeddings and text classification with nave base and familiarize yourself with the fundamentals of libraries like spacy and NLTK and work on end toend NLP project and talking about computer vision track focus on basic image processing techniques like filtering edge detection image scaling and rotation. Utilize libraries like OpenCV and build upon the CNN knowledge from the previous month. Practice data prep-processing and augmentation. So by the end of this month, you should have a solid foundation in your chosen specialization ready to embark on your AI engineering career. So in conclusion, adopting AI is more than just a trend. It's a strategic move that can transform your organization approach to machine learning. So do check out our professional certificate course in AI and machine learning which is designed to equip you with the skills needed to excel in this dynamic field. Through live classes led by industry experts, master classes from IIT Kpur faculty and hands-on projects, you'll gain a competitive edge in AI. To enroll in this program, candidates should have a bachelor's degree with a 50% or higher average, preferably two or more years of work experience and foundational knowledge in programming and mathematics. Check out the course link in the description box below. So that's all for today's video on becoming an AI engineer. We hope you found this guide valuable and informative. LLMs, if you ever wondered how machine learning can now understand and generate humanlike text, you are in the right place. From chat boards like Chat GP to AI assistant that powers search engines, LLMs are transforming how we interact with technology. One of the most exciting advancement in this space is Google's Gemini or OpenAI Charging large language model designed to push the boundaries of what AI can achieve. In this video, we will explore what LLMs are, how they work, and why models like Gemini are critical for the future of AI. Google Gemini is part of a new wave of AI models that are smarter, faster, and more efficient. It is designed to understand context better, offer more accurate responses and integrate deeply into service like Google search and Google assistant providing more humanlike interactions. So we will break down the science behind LLMs including their massive training data set transformer architecture and how models like Gemini use deep learning innovation to change industries. Plus, we will compare Google Gemini to other popular LMS such as OpenAI chatting models, showing how each of these technologies is used to power chat bots, virtual assistants, and other AIdriven application. By end of this video, you will have a clear understanding of how large language models like Gemini work, their key features, and what they mean for their future AI. Don't forget to like, subscribe, and hit the bell icon to never miss any update from Simply Learn. So, what are the large language models? Large language models like CH GPD4 generative pre-trained transformer 4 O and Google Gemini are sophisticated AI system designed to comprehend and generate humanlike text. These models are built using deep learning techniques and are trained on vast data set collected from the internet. They leverage self attention mechanism to analyze relationship between words or tokens allowing them to capture context and produce coherent relevant responses. LLMs have significant application including powering virtual assistant, chatboards, content creation, language translation and supporting research and decision making. Their ability to generate fluent and contextually appropriate text has advanced natural language processing and improved human computer interaction. So now let's see what are large language model used for. Large language models are utilized in scenarios with limited or no domain specific data available for training. These scenarios include both few short and zero short training approaches which rely on the model's strong inductive bias and its capability to derive meaningful representation from a small amount of data or even no data at all. So now let's see how are large language model train. Large language models typically undergo pre-training on a board. All encompassing data set that shares statical similarities with the data set specific to the target task. The objective of pre-training is to enable the model to require highlevel feature that can later be applied during the fine-tuning phase for a specific task. So there are some training processes of LLM which involves several steps. The first one is text prep-processing. The textual data is transformed into a numerical representation that the LLM model can effectively process. This conversion may be involve techniques like tokenization, encoding and creating input sequences. The second one is random parameter initialization. The model's parameter are initialized randomly before the training process begins. The third one is input numerical data. The numerical representation of the text data is fed into the model of processing. The model's architecture typically based on transformers allows it to capture the conceptual relationship between the words or tokens in the next. The fourth one is loss function calculation. A loss function calculation measure the discrepancy between the model's prediction and the actual next word or token in a syntax. The LLM model aims to minimize this loss during training. The fifth one is parameter optimization. The model's parameter are registered through optimization technique. This involves calculating gradient and updating the parameters accordingly gradually improving the model's performance. The last one is iterative training. The training process is repeated over multiple iteration or epochs until the model's output achieve a satisfactory level of accuracy on that given task or data set. By following this training process, large language model learn to capture linguistic patterns, understand context and generate coherent responses, enabling them to excel at various language related tasks. The next topic is how do large language models work. So large language models leverage deep neural network to generate output based on patterns learned from the training data. Typically a large language model adopts a transformer architecture which enables the model to identify relationship between words in a sentence irrespective of their position in the sequence. In contrast to RNNs that rely on recurrence to capture token relationship transformer neural network employ self attention as their primary mechanism. Self attention calculates attention scores that determine the importance of each token with respect to the other token in the text sequence. Facilating the modeling of intricate relationship within the data. Next let's see application of large language models. Large language models have a wide range of application across various domains. So here are some notable application. The first one is natural language processing NLP. Large language models are used to improve natural language understanding tasks such as sentiment analysis, named entity recognition, text classification, and language modeling. The second one is chatbot and virtual assistant. Large language models power conversational agents, chatbots, and virtual assistant providing more interactive and humanlike user interaction. The third one is machine translation. LA language models have been used for automatic language translation enabling text translation between different languages with improved accuracy. The fourth one is sentiment analysis. LLM can analyze and classify the sentiment or emotion expressed in a piece of text which is valuable for market research, brand monitoring and social media analysis. The fifth one is content recommendation. These models can be employed to provide personalized content recommendations enhancing user experience and engagement on platforms such as news website or streaming services. So these application highlight the potential impact of large language models in various domains for improving language understanding automation. >> Why reinforcement learning? Training a machine learning model requires a lot of data which might not always be available to us. Further, the data provided might not be reliable. Learning from a small subset of actions will not help expand the vast realm of solutions that may work for a particular problem. And you can see here we have the robot learning to walk. Um, very complicated setup when you're learning how to walk. And you'll start asking questions like, if I'm taking one step forward and left, what happens if I pick up a 50 lb object? How does that change how a robot would walk? These things are very difficult to program because there's no actual information on it until the it's actually tried out. Learning from a small subset of actions will not help expand the vast realm of solutions that may work for a particular problem. And we'll see here it learned how to walk. This is going to slow the growth that technology is capable of. Machines need to learn to perform actions by themselves and not just learn off humans. And you see the objective climb a mountain. Real interesting point here is that as human beings, we can go into a very unknown environment and we can adjust for it and kind of explore and play with it. Most of the models, the non-reinforcement models in computer u machine learning aren't able to do that very well. Uh there's a couple of them that can be used or integrated. see how it goes is what we're talking about with reinforcement learning. So what is reinforcement learning? Reinforcement learning is a subbranch of machine learning that trains a model to return an optimum solution for a problem by taking a sequence of decisions by itself. Consider a robot learning to go from one place to another. The robot is given a scenario must arrive at a solution by itself. The robot can take different paths to reach the destination. it will know the best path by the time taken on each path. It might even come up with a unique solution all by itself. And that's really important as we're looking for unique solutions. Uh we want the best solution, but you can't find it unless you try it. So we're looking at uh our different systems, our different model. We have supervised versus unsupervised versus reinforcement learning. And with the supervised learning, that is probably the most controlled environment. uh we have a lot of different supervised learning models where there's linear regression, neural networks, um there's all kinds of things in between, decision trees. The data provided is labeled data with output values specified. And this is important because we talk about supervised learning, you already know the answer for all this information. You already know the picture has a motorcycle in it. So you're supervised learning. you already know that um the outcome for tomorrow for you know going back a week you're looking at stock you can already have like the graph of what the next day looks like so you have an answer for it and you have labeled data which is used you have an external supervision and solves problems by mapping labeled input to known output so very controlled unsupervised learning and unsupervised learning is really interesting because it's now taking part in many other models. They start with an you can actually insert an unsupervised learning model um in almost either supervised or reinforcement learning as part of the system which is really cool. Uh data provided is unlabeled data. The outputs are not specified. Machine makes its own predictions used to solve association with clustering problems. Unlabeled data is used. No supervision. Solves problems by understanding patterns and discovering output. Uh so you can look at this and you can think um some of these things go with each other. They belong together. So it's looking for what connects in different ways. And there's a lot of different algorithms that look at this. Um when you start getting into those are some really cool images that come up of what unsupervised learning is. How it can pick out say uh the area of a donut. One model will see the area of the donut and the other one will divide it into three sections based on its location versus what's next to it. So there's a lot of stuff that goes in with unsupervised learning. And then we're looking at reinforcement learning. Probably the biggest industry in today's market uh in machine learning or growing market. It's very it's very infant stage uh as far as how it works and what it's going to be capable of. The machine learns from its environment using rewards and errors used to solve rewardbased problems. No predefined data is used. No supervision. Follows trail and error problem solving approach. Uh so again we have a random at first you start with a random I try this it works and this is my reward. Doesn't work very well maybe or maybe doesn't even get you where you're trying to get it to do. And you get your reward back and then it looks at that and says well let's try something else. and it starts to play with these different things finding the best route. So let's take a look at important terms in today's reinforcement model and this has become pretty standardized over the last uh few years. So these are really good to know. We have the agent uh agent is the model that is being trained via reinforcement learning. So this is your actual u entity that has however you're doing it whether you're using a neural network or a Q table or whatever combination thereof. This is the actual agent that you're using. This is the model and you have your environment. Uh the training situation that the model must optimize to is called its environment. Uh and you can see here I guess we have a robot who's trying to get a chest full of gyms or whatever. And that's the output. And then you have your action. This is all possible steps that can be taken by the model and it picks one action and you can see here it's picked three different uh routes to get to the chest of diamonds and gems. We have a state the current position condition returned by the model. And you could look at this uh if you're playing like a video game. This is the screen you're looking at. Uh so when we go back here uh the environment is the whole game board. So, if you're playing one of those Mobius games, you might have the whole game board going on. Uh, but then you have your current position. Where are you on that game board? What's around that? What's around you? Um, if you were talking about a robot, the environment might be moving around the yard, where it is in the yard, and what it can see, what input it has in that location. That would be the current position condition returned by the model. And then the reward uh to help the model move in the right direction. It is rewarded. Points are given to it to appraise some kind of action. So yeah, you did good or if uh didn't do as good trying to maximize the reward and have the best reward possible. And then policy. Policy determines how an agent will behave at any time. It acts as a mapping between action and present state. This is part of the model. What what what is your action that you're you're going to take? what's the policy you're using to have an output from your agent. One of the reasons they separate a policy as its own entity is that you usually have a prediction um of a different options and then the policy well how am I going to pick the best based on those predictions I'm going to guess at different options and we'll actually weigh those options in and find the best option we think will work. Uh, so it's a little tricky, but the policy thing is actually pretty cool how it works. Let's go ahead and take a look at a reinforcement learning example. And just in looking at this, we're going to take a look uh consider what a dog um that we want to train. Uh so the dog would be like the agent. So you have your your puppy or whatever. Uh and then your environment is going to be the whole house or whatever it is where you're training them. And then you have an action. We want to teach the dog to fetch. So action equals fetching. Uh and then we have a little biscuit. So we can get the dog to perform various actions by offering incentives such as a dog biscuit as a reward. The dog will follow a policy to maximize this reward and hence will follow every command and might even learn new actions like begging by itself. Uh so you have be you know so we start off with fetching. It goes oh I get a biscuit for that. it tries something else and you get a handshake or begging or something like that and it goes oh this is also reward-based and so it kind of explores things to find out what will bring it as biscuit and that's very much like how a reinforced model goes is it uh looks for different rewards how do I find can I try different things and find a reward that works the dog also will want to run around and play and explore it environment uh this quality of model is called exploration so there's a little randomness going on in exploration and explores new parts of the house. Climbing on the sofa doesn't get a reward. In fact, it usually gets kicked off the sofa. So, let's talk a little bit about Marov's decision process. Uh Marov's decision process is a reinforcement learning policy used to map a current state to an action where the agent continuously interacts with the environment to produce new solutions and receive rewards. And you'll see here's all of our different uh uh vocabulary we just went over. We have our reward, our state, our agent, our environment interaction. And so even though the environment kind of contains everything um that you you really when you're actually writing the program, your environment is going to put out a reward and state that goes into the agent. Uh the agent then looks at this uh state or it looks at the reward usually um first and it says okay I got rewarded for whatever I just did or I didn't get rewarded and then it looks at the state and then it comes back and if you remember from policy the policy comes in um and then we have a reward. The policy is that part that's connected at the bottom. And so it looks at that policy and it says, "Hey, what's a good action that will probably be similar to what I did?" Or um uh sometimes they're completely random, but what's a good action that's going to bring me a different reward? So, taking the time to just understand these different pieces as they go is pretty important in most of the models today. Um, and so a lot of them actually have templates based on this that you can pull in and start using. Um, pretty straightforward as far as once you start seeing how it works. Uh, you can see your environment send it says, "Hey, this is the agent did this. If you're a character in a game, this happened and it shoots out a reward in a state." The agent looks at the reward, looks at the new state, and then takes a little guess and says, "I'm going to try this action." And then that action goes back into the environment. it affects the environment. The environment then changes depending on what the action was and then it has a new state and a new reward that goes back to the agent. So in the diagram shown, we need to find the shortest path between node A and D. Each path has a reward associated with it and the path with a maximum reward is what we want to choose. The nodes A, B, C, Denote the nodes to travel from node uh A to B is an action. Reward is the cost of each path and policy is each path taken. And you can see here A can go uh to B or A can go to C right off the bat or it can go right to D. And if you explored all three of these uh you would find that A going to D was a zero reward. Um A going to C and D would generate a different reward. Or you could go AC B D. There's a lot of options here. Um and so when we start looking at this diagram, you start to realize that even though uh today's reinforced learning models do really good at um finding an answer, they end up trying almost all the different directions you see. And so they take up a lot of work uh or a lot of processing time for reinforcement learning. They're right now in their infant stage and they're really good at solving simple problems and we'll take a look at one of those in just a minute in a tic-tac-toe game. Uh but you can see here uh once it's gone through these and it's explored, it's going to find the AC D is the best reward. It gets a full 30 points for it. So let's go ahead and take a look at a reinforcement learning demo. Uh and in this demo, we're going to use reinforcement learning to make a tic-tac-toe game. You'll be playing this game against the machine learning model. And we'll go ahead and we're doing it in Python. So, let's go ahead and go through um I always uh not always actually have a lot of Python tools. Let's go through um Anaconda, which will open up a Jupyter notebook. Seems like a lot of steps, but it's worth it to keep all my stuff separate. And it's also has a nice display when you're in the Jupyter notebook for doing Python. So, here's our Anaconda Navigator. I open up the notebook, which is going to take me to a web page. And I've gone in here and created a new uh Python folder. In this case, I've already done it and enabled it. Change the name to tic-tac-toe. Uh, and then for this example, uh, we're going to go ahead and import a couple things. We're going to, um, import numpy as np. We'll go ahead and import pickle. Numpy, of course, is our number array. And then, uh, pickle is just a nice way sometimes for storing, uh, different information, uh, different states that we're going to go through on here. Uh, and so we're going to create a class called state. I'm going to start with that. And there's a lot of uh lines of code to this uh class that we're going to put in here. Don't let that scare you too much. There's not as much here. Um it looks like there's going to be a lot here, but there really is just a lot of setup going on in the in our class state. And so we have up here, we're going to initialize it. Um we have our board. Um it's a tic-tac-toe board, so we're only dealing with nine spots on the board. Uh we have player one, player two, uh is end. We're going to create a board hash. Uh we'll look at that in just a minute. We're just going to store some information in there. Symbol of player equals 1. Um so there's a few things going on as far as the initialization. Uh then something simple. We're just going to get the hash um of the board. We're going to get the information from the board on there, which is uh columns and rows. We want to know when a winner occurs. Uh, so if you get three in a row, that's what this whole section here is for. Uh, let me go ahead and scroll up a little bit. And you can get a copy of this code if you send a note over to SimplyLearn. We'll send you over um this particular file and you can play with it yourself and see how it's put together. I don't want to spend a huge amount of time on this uh because this is just some real general Python coding. Uh but you can see here we're just going through um all the rows and you add them together and if it equals three three in a row same thing with columns u diagonal. So you got to check the diagonal. That's what all this stuff does here is it just goes through the different areas. Actually let me go ahead and put there we go. Um and then it comes down here and we do our sum and it says true uh minus three. Just says did somebody win or is it a tie? So, you got to add up all the numbers on there anyway, just in case they're all filled up. And next, we also need to know available positions. Um, these are ones that don't no one's ever used before. This way, when you try something or the computer tries something, uh, it's not going to give it an illegal move. That's what the available positions is doing. Uh, then we want to update our state. And so, you have your position going in. We're just sending in the position that you just chose. And you'll see there's a little user interface we put in there. You can p pick the row and column in there. And again, I mean, this is a lot of code. Uh so really, it's kind of a thing you'd want to go through and play with a little bit and just read through it, get a copy of it. Uh great way to understand how this works. And here is a given reward. Um so we're going to give a reward result equals self winner. This is one of the hearts of what's going on here uh is we have result self.winner. So if there's a winner, then we have a result. If the result equals one, here's our feedback. Uh if it doesn't equal one, then it gets a zero. So it only gets a reward in this particular case if it wins. And that's important to know because different uh systems of reinforced learning do rewarding a lot differently depending on what you're trying to do. This is a very simple example with a 3x3 board. Imagine if you're playing a video game. Uh certainly you only have so many actions, but your environment is huge. You have a lot going on in the environment. And suddenly a reward system like this is going to be just um is going to have to change a little bit. It's going to have to have different rewards and different setup. And there's all kinds of advanced ways to do that as far as weighing. You add weights to it. And so they can add the weights up depending on where the reward comes in. So it might be that you actually get a reward. In this case, you get the reward at the end of the game. And I'm spending just a little bit of time on this because this is an important thing to note. But there's different ways to add up those rewards. It might have like if you take a certain path, um the first reward is going to be weighed a little bit less than the last reward because the last reward is actually winning the game or scoring or whatever it is. So, this reward system gets really complicated on some of the more advanced uh setups. Um, in this case though, you can see right here that they give a um a 0.1 and a.5 reward um just for getting a picking the right value and something that's actually valid instead of picking an invalid value. So, rewards again, that's like key. That's huge. How do you feed the rewards back in? Uh then we have a board reset. That's pretty straightforward. It just goes back and resets the board to the beginning because it's going to try out all these different things while it's learning. It's going to do it by trial and error. So, you have to keep resetting it. And then, of course, there's the play. We want to go ahead and play uh rounds equals 100. Depends on what you want to do on here. Um you can set this different. You obviously set that to higher level, but this is just going to go through and you'll see in here uh that we have player one and player two. This is this is the computer playing itself. Uh, one of the more powerful ways to learn to play a game or even learn something that isn't a game is to have two of these models that are basically trying to beat each other. And so they always they keep finding explore new things. This one works for this one. So this one tries new things. It beats this. We've seen this in um chess I think was a big one where they had the two players in chess with reinforcement learning. uh is one of the ways they train one of the top um computer chess playing algorithms. Uh so this is just what this is. It's going to choose an action. It's going to try something and the more it try stuff um the more we're going to record the hash. We actually have a board hash where itself get the hash set up on here where it stores all the information and then once you get to a win, one of them wins, it gets the reward. Uh then we go back and reset and try again. And then kind of the fun part we actually get down here is uh we're going to play with a human. So we'll get a chance to come in here and see what that looks like when you put your own information in. And then it just comes in here and does the same thing it did above. It gives it a reward for its things um or sees if it wins or ties. Um looks at available positions, all that kind of fun stuff. And then finally, we want to show the board. Uh so it's going to print the board out each time. Really um as an integration is not that exciting. What's exciting uh in here is one looking at this reward system. Whoops. Play one more up. The reward system is really the heart of this. How do you reward the different uh setup and the other one is when it's playing it's got to take an action. And so what it chooses for an action is also the heart of reinforcement learning. How do we choose that action? And those are really key to right now where reinforcement learning is uh in today's uh technology is uh figuring this out. How do we reward it and how do we guess the next best action? So we have our uh environment and you can see the environment is we're going to be or the state uh which is kind of like what's going on. We're going to return the state depending on what happens. And we want to go ahead and create our agent. Uh in this case, our player. So each one is let me go and grab that. And so we look at a class player. Um this is where a lot of the magic is really going on is what how is this player figuring out how to maneuver around the board? And then the board of course returns a state uh that it can look at and a reward. Uh so we want to take a look at this. We have a name uh self state. This is class player. And when you say class player, we're not talking about a human player. We're talking about u just a uh the computer players. And this is kind of interesting. So remember I told you depending on what you're doing, there's going to be a decay gamma. Um explore rate. Uh these are what I'm talking about is how do we train it? Um as you try different moves, it gets to the end. The first move is important, but it's not as as important as the last one. And so you could say that the last one has the heaviest weight. And then as you as you get there, the first one, let's see, the first move gives you a five reward, the second gives you a two reward, and the third one gives you a 10 reward because that's the final ending. You got it. The 10's going to count more than the first step. Uh, and here's our, uh, we're going to, you know, get the board information coming in and then choose an action. This was the second part that I was talking about that was so important. Uh so once you have your training going on, we have to do a little randomness. And you can see right here is our NP random u uniform. So it's picking out a random number. Take a random action. And this is going to just pick which row and which column it is. Um and so choosing the action. This one you can see we're just doing random states. uh choice, length of positions, action position, and then it skips in there and takes a look at the board uh for P and positions. And you it's actually storing the different boards each time you go through so it has a record of what it did so it can properly weigh the values. And this simply just appends a hash state. What's the last state? Pinned it to the uh to our states on here. Here's our feedback reward. the reward comes in and it's going to take a look at this and say is it none uh what is the reward and here is that formula remember I was telling you about up here um that was important because it has decay gamma times the reward this is where as it goes through each step and this is really important this is this is kind of the heart of this of what I was talking about earlier uh you have step one and This might have a a reward of two. You have step two. I probably should have done ABC. This has a step three. Uh step four. So on till you get to step in. And this might have a reward of 10. Uh so reward of 10. We're going to add that. But we're not adding uh let's say this one right here. Uh let's say this reward here right before 10 was um let's say it's also 10. That just makes the the uh math easy. So we had 10 and 10. Uh we had 10. This is 10 and 10 in whatever it is. But this time it's 0.9. Uh so instead of putting a full 10 here, we only do nine. That's uh 0.9 times 10. And so this formula um as far as the decay times the reward minus the cell state value uh it basically adds in it says here's one or here's two. I'm sorry I should have done this ABC would have been easier. Uh so the first move goes in here and it puts two in here. Uh then we have our self uh setup on here. You can see how this gets pretty complicated in the math, but this is really the key is how do we train our states and we want the the final state, the win to get the most points. If you win, you get most points. Um, and the first step gets the least amount of points. So, you're really training this almost in reverse. You're retraining you're training it from the last place where you have like it says okay this is now where need to sum up my rewards and I want to sum them up going in reverse and I want to find the answer in reverse kind of an interesting uh uh play on the mind when you're trying to figure this stuff out and of course we want to go ahead and reset the board down here uh save the policy load policy these are the different things that are going in between the agent and the state to figure out what's going on let's go ahead load that up. And then finally, we want to go ahead and create a human player. And the human player is going to be a little different uh in that uh you choose an action row and column. Here's your action. Uh if action is if action in positions, meaning positions that are available, uh you return the action. If not, it just keeps asking you until you get an action that actually works. And then we're going to go ahead and append to the hash state which uh we don't need to worry about because it returns the action up here and feed forward. Uh again this is because it's a human. Um at the end of the game bat propagate and update state values. This part isn't being done because it's not programming uh the model. Uh the model is getting its own rewards. So we've gone ahead and loaded this in here. Uh so here's all our pieces. And the first thing we want to do is set up uh P1 player one uh P2 player two. And then we're going to send our players to our state. So now it has P1 P2 and it's going to play and it's going to play 50,000 rounds. Now we can probably do a lot less than this and it's not going to get the full results. In fact, you know what? Uh, let's go ahead and just do five. Um, just to play with it because I want to show you something here. Oops. Somewhere in there I forgot to load something. There we go. I must have forgot to run this run. Oops, forgot a reference there for the board rows and columns 3x3. U there is actually in the state it references that. We just tack it on on the end. It was supposed to be at the beginning. Uh, so now I've only set this up with um, see where are we going here? I've only set this up to train five times. And the reason I did that is we're going to uh, come in and actually play it. And then I'm going to change that and we can see how it differs on there. There we go. It didn't make it through a run. And we're going to go ahead and save the policy. Um, so now we have our player one and our player two policy. Uh, the way we set it up, it has two separate policies loaded up in there. And then we're going to come in here and we're going to do uh, player one is going to be the computer experience rate zero, load policy one, human player human. And we're going to go ahead and play this. Now remember, I only went through it um, uh, just one round of training. In fact, minimal training. And so it puts an X there. And I'm going to go ahead and do row zero, column one. You can see this is very uh basic on here. And so I put in my zero. And then I'm going to go zero, block it, zero, zero. And you can see right here, it let me win. Uh just like that, I was able to win. Zero two. And woo, human wins. So I only trained it five times. We're going to run this again. And this time, uh, instead of five, let's do 5,000 or 50,000. I think that's what the guys in the back had. And this takes a while to train it. This is where reinforcement learning really falls apart. Look how simple this game is. We're talking about a 3x3 set of columns. And so for me to train it on this um I could do a Q table which would take which would go much quicker. Um you could build a quick Q table with almost all the different options on there and uh you would probably get a the same result much quicker. We're just using this as an example. So when we look at reinforcement learning, you need to be very careful what you apply it to. It sounds like a good deal until you do like a large neural network where you're doing um you set the neural network to a learning increment of one. So every time it goes through it learns and then you do your actions. So you pick from the learning uh setup and you actually try actions on the learning setup until you get the what you think is going to be the best action. So you actually feed what you think is right back through the neural network. There's a whole layer there which is really fun to play with. and then it has an output. Well, think of all those processes. I mean, that is just a huge amount of work it's going to do. Uh let's go ahead and skip ahead here. Give it a moment. It's going to take a a minute or two to go ahead and run now to train it. Uh we went ahead and let it run and it took a while. This this took um I got a pretty powerful processor and it took about five minutes plus to run it. and we'll go ahead and uh run our player setup on here. Oops, it brought in the last Whoops, it brought in the last round. So, give me just a moment to reddo the policy save. There we go. I forgot to save the policy back in there and then go ahead and run our player again. So, we've saved the policy and then we want to go ahead and load the policy for P1 as a computer. And we can see the computer's gone in the bottom right corner. I'm going to go ahead and go uh one one which is the center and it's gone right up the top. And if you have ever played tic-tac-toe, you know the computer has me. Uh but we'll go ahead and play it out. Row zero, column two. There it is. And then it's gone here. And so I'm going to go ahead and go row 0 one two. No, 01. There we go. And column zero. That's where I wanted. Oh, and it says I Okay, you your action. There we go. Boom. Uh, so you can see here we've got a didn't catch the win on this. It said tie. Um, kind of funny that it didn't catch the win on there. But if we play this a bunch of times, you'll find it's going to win more and more. The more we train it, the more the reinforcement happens. This lengthy training process uh is really the stopper on reinforcement learning. As this changes, reinforcement learning will be one of the more powerful uh packages evolving over the next decade or two. In fact, I would even go as far as to say it is the most important uh machine learning tool and artificial intelligence tool out there as it learns not only a simple tic-tac-toe board, but we start learning environments. And the environment would be like in language, if you're translating a language or something from one language to the other, so much of it is lost if you don't know the context. it's in what's the environments it's in. And so being able to attach environment and context and all those things together is going to require reinforcement learning to do. So again, if you want to get a copy of the tic-tac-toe board, it's kind of fun to play with. Uh run it, you can test it out, you can do um you know, test it for different uh uh values. You can switch from P1 computer uh where we loaded the policy one to load the policy two and just see how it varies. There's all kinds of things you can do on there. So, what is Q-learning? Q-learning is reinforcement learning policy which will fill the next best action given a current state. It chooses this action at random and aims to maximize the reward. And so, you can see here's our standard reinforcement learning graph. Um, by now if you're doing any reinforcement learning, you should be familiar with this where you have your agent, your agent takes an action, the action affects the environment, and then the environment sends back the reward or the feedback and the state it's the new state the agent's in. Where is it at on the chessboard? Where's it at in the video game? Um, if your robots's out there picking trash up off the side of the road, where is it at on the road? Consider an ad recommendation system. Usually when you look up a product online, you get ads which will suggest the same product over and over again. Using Q-learning, we can make an ad recommendation system which will suggest related products to our previous purchase. The reward will be if user clicks on the suggested product. And again, you can see um you might have a lot of products on uh your web advertisement or your pages, but it's still not a float number. It's still a set number. And that's something to be aware of when you're using Q-learning. And you can see here that if you have a 100 people clicking on ads and you click on one of the ads, it might go in there and say, "Okay, this person clicked on this ad. What is the best set of ads based on clicking on this ad or these two ads afterwards, based on where they are browsing? So let's go and take a look at some important terms when we talk about Q-learning. Uh we have states the state S represents the current position of an agent in an environment. Um the action the action A is the step taken by the agent when it is particular state. Rewards for every action the agent will get a positive or negative reward. And again, uh, when we talk about states, we're usually not with when you're using a Q table, you're not usually talking about float variables. You're talking about true false. Um, and we'll take a closer look at that in a second. And episodes when an agent ends up in a terminating state and can't take a new action. Uh, this might be if you're playing a video game, your character stepped in and is now dead or whatever. uh Q values used to determine how good an action A taken at a particular state S is Q A of S and temporal difference a formula used to find the Q value by using the value of the current state and action and previous state and action and very I mean there's Bellman's equation which basically is the equation that kind of uh covers what we just looked at in all those different terms. The Bellman equation is used to determine the values of a particular state and deduce how good it is to be in take that state. The optimal the optimal state will give us the highest optimal value factor influencing Q values. The current state and action that's your SA. So your current state and your action uh and then you have your previous state in action which is your S um I guess prime. I'm not sure how they how they reference that S prime A prime. So this is what happened before. Uh then you have a reward for action. So you have your R reward and you have your maximum expected future reward. And you can see there's also a learning rate put in there and a discount rate. Uh so we're looking at these just like any other model. We don't want to have an absolute um final value on here. We don't want it to. If you do absolute values instead of taking smaller steps, you don't really have that approach to the solution. You just have a jump and then pretty soon if you jump one solution out, that's what's going to be the new solution. Whichever one jumps up really high first. Um kind of ruining the whole idea of doing a random selection. And I'll go into the random selection in just a second. Steps in Q-learning. Step one, create an initial Q table with all values initialized to zero. Again, we're looking at 01. Uh, so are you, you know, here's our action. We start, we're an idle. We took a wrong action. We took a correct action and end. And then we have our um actions fetching, sitting, and running. And of course, we're just using the dog example. And choose an action and perform it. Update values in the table. And of course, when we're choosing an action, we're going to kind of do something random and just randomly pick one. So, you start out and you sit and you have then a um then depending on that um um action you took, you can now update the value for sitting after you start from start to sitting, get the value of the reward and calculate the val the value Q value using the Bellman equation. And so now we attach a reward to sitting. And when we attach all those rewards, we continue the same until the table's filled with or an episode ends. And and I mentioned I was going to come back to the random side of this. And there's a few different formulas they use for the random uh setup to pick it. I usually let whatever Q model I'm using do their standard one because someone's usually gone in and done the math uh for the optimal uh spread. Uh but you can look at this. If I have running has a reward of 10, sitting has a reward of seven, fetching has a reward of five. Um, just kind of without doing like a a means, you know, using the bell curve for the means value. And like I said, there's some math you can put in there to pick um so that you're more like so that running has even a higher chance. Uh, but even if you were just going to do an average on this, you could do an average, a random number by adding them all together. Uh so you get 10 + 7 + 5 is 22. You could do 0 to 22 or 0 to 21 but 1 to 22. 1 to 5 would be fetching uh and so forth. You know the last 10. So you can just look at this as what percentage are you going to go for that particular option. Um and then that gets your random setup in there. And then as you slowly increment these up, uh you see that uh u if you're idle, u where's one? Here we go. Sitting at the end, if you're at the end of wherever you're at, sitting gets a reward of one. Um where's a good one on here? Oh, wrong action. Running for a wrong action gets almost no reward. So that becomes very very less likely to happen, but it still might happen. It still might have a percentage of coming up. And that's where the random programming and Q-learning comes in. The below table gives us an idea of how many times an action has been taken and how positively correct action or negatively wrong action it is going to affect the next state. So let's go ahead and dive in and pull up a little piece of code and see what this looks like um in Python. Uh in this demo we'll use Q-learning to find the shortest path between two given points. >> If getting your learning started is half the battle, what if you could do that for free? Visit Skillup by SimplyLearn. Click on the link in the description to know more. >> If you've seen my videos before, um I like to do it in the uh Anaconda Jupiter notebook um setup just because it's really easy to see and it's a nice demo. Uh and so here's my Anaconda. This one I'm actually using a Python 36 environment that I set up in here. And we'll go ahead and launch the Jupyter notebook on this. And once we're in our Jupyter notebook, uh, which has the kernel loaded with Python 3, we'll go ahead and create a new Python 3 uh, folder in here. And we'll call this, uh, Q learning. And to start this demo, let's go ahead and import our uh, numpy array. We'll just run that. So, it's imported. And like a lot of these uh, model programs, when you're building them, you spend a lot of time putting it all together. Um and then you end up with this really short answer at the end. Uh and we'll we'll take a look at that as we come into it. So we we go ahead and start with our location to state. Uh so we have um L1, L2. These are our nine locations 1 to nine. And then of course the state is going to be 0 1 2 3 4. It's just a mapping of our location to a integer on there. And then we have our actions. Our actions are simply uh moving from uh one location to another. So I can go to I can go to location zero. I can go to location 1 2 3 4 5 6 7 8. Uh so these are my actions. I can choose these are the locations of our state. And if you remember earlier, I mentioned uh um that the limitation is that you you don't want to put in um a continually growing table because you can actually create a dynamic Q table where you continually add in new values as they arise because um if you have float values, it just becomes infinite and then your memory in your computer's gone or you know it's not going to work. At the same time, you might think, well, that kind of really limits the the Q uh T learning setup, but there are ways to use it in conjunction with other systems. And so you might look at uh well I do um I've been doing some work in stock um and one of the questions that comes out is to buy or sell the stock and the stake coming in might be um you might take it and create what we call buckets um where anything that you predict is going to return more than a certain amount of money um the error for that stock that you've had in the past you put those in bucket And suddenly you start putting the creating these buckets you realize you do have a limited amount of information coming in. You no longer have a float number. You now have um bucket 1 2 3 and four. And then you can take those buckets, put them through a a Q-learning table and come up with the best action. Which stock should I buy? It's like gambling. Stock is pretty much gambling if you're doing day trading. You're not doing long-term u investments. And so you can start looking at it like that. A lot of the um current feeds say that the best algorithms used for day traders where you're doing it on your own is really to ask the question, do I want to trade the stock? Yes or no? And now you have it in a Q-learning table and now you can take it to that next level and you can see where that can be a really powerful tool at the end of doing a basic linear regression model or something. um what is the best investment and you start getting the best reward on there. Uh and so if we're going to have rewards, these rewards we just create um it says uh if basically if you're uh this should match our Q table because it's going to be uh you have your state and you have your action across the top if you remember from the dog. And so we have whatever state we're in going down and then the next action and what the reward is for it. Um, and of course, if you were actually doing a u something more connected, your reward would be based on uh the actual environment it's in. And then we want to go ahead and create a state to location uh so we can map the indexes. So just like we defined our rewards, uh we're going to go ahead and do state to location. Um and you can see here it's a a dictionary setup for location state and locationto state with items and We also need to um define what we want for learning rates. Uh you remember we had our two different rates um as far as like learning from the past and learning from the current. So we'll go ahead and set those to uh 75 and the alpha set to 0.9. And we'll see that when we do the formula. And of course any of this code uh send a note to our SimplyLearn team. They'll get you a copy of this code on here. Let's go ahead and pull. There we go. The new next two sections. Um, since we're going to keep it short and sweet. Here we go. So, let's go ahead and create our agent. Um, so our agent is going to have our initialization where we send it all the information. Uh, we'll define our self gamma equals gamma. We could have just set the gamma rate down here instead of uh submitting it. It's kind of nice to keep them separate because you can play with these numbers uh our self-alpha. Um then we have our location state. We'll set that in here. Um we have our choice of actions. Um we're going to go ahead and just embed the rewards right into the agent. So obviously this would be coming from somewhere else uh instead of from uh self-generated. And then a self state to location equals our state to location uh dictionary. And we go ahead and create a Q-learning table. And I went ahead and just set the Q-learning table up to um uh 0 to zero. What what what the setup is uh location to state. How many of them are there? Uh and this just creates an array of zero to zero setup on there. And then the big part is the training. We have our rewards new equals a copy of self.rewards. rewards ending state equals the self-location state end location. So this is whatever we end up at. Rewards new equals ending state plus ending state equals 999. Just kind of goes to a dead end. And we start going through iterations and we'll go ahead. Um let's do this. Uh so this we're going to come back and we're going to call call it on here. Uh, let me just erase that. Switch it to an arrow. There we go. Uh, so what we're doing is we're going to send in here to train it. We're going to say, hey, um, I want to iterate through this a thousand times and see what happens. Now, this part would actually be instead of iterating, you might have your external environment and they're going back and forth and you iterate through outside of here. Uh, but just for ease of use, our agent's going to come in here and iterate through this. Sometimes I'll put this iteration in here and I'll have it call the environment and say, "Hey, this is what I did. What's the next state?" And the environment does its thing right in here as I iterate through it. Uh, and then we want to go ahead and pick a random state to start with. That's what's going on here. You have to start somewhere. Um, and then you have your playable actions. We're going to start with just an empty thing for playable actions and we'll fill that up. So that's what choices I have. And so we're going to iterate through the rewards matrix to get the states uh directly reachable from the randomly chosen current state. Assign those states to a list named playable actions. And so you can see here we have uh range nine. I usually use length of whatever I'm looking at uh which is our locations or states as they are. uh we have a reward. So we want to look at the current the rewards uh the new reward is our uh is in our chart here of rewards new uh current state um plus J. Uh J being what is the next state we want to try. And so we go ahead and do our playable actions and we append J. And so we're doing is we're randomly trying different things in here to see what's going to generate a better reward. And then of course we go ahead and choose our next state. Uh so we have our random choice playable actions. And if you remember I mentioned on this let me just go ahead and uh whoops let's do a free form. When we were talking about the next state uh this right here just does a random selection. Instead of a random uh selection, you might do something where uh whatever the best selection is, which might be option three here. And then so you can see that it might use a bell curve and then option two over here might have a bell curve like this. Oops. And we start looking at these averages and these spreads. Um or we can just add them all together and pick the one that kind of goes in all of those. Uh so those are some of the options we have in here. We just go with a random choice. Uh that's usually where you start, play with it. Um and then we have our reward section down here. And so we want to go ahead and find well in this case the temporal difference. Uh so you have your rewards new plus the self gamma. And this is the formula we were looking at. This is Bellman's equation here. Uh so we have our current value, our learning rate, our discount rate involved in there, the reward system coming in for that. Uh and we can add it all together. This is of course our uh maximum expected future setup in here. Uh so this is all of our our Bellman's equation that we're looking at here. And then we come up in here and we update our Q table. That's all this is on this one. Uh that's right here. We have um self Q current state next state and we add in our um alpha because we don't want to we don't want to train all of it at once in case there's slight differences coming in there. We want to slowly approach the answer. Uh and then we have our route equals the start location and next location equals start location. So we're just incrementing. We took a step forward. And then finally, remember I was telling you how uh we're going to do all this and just have some simple thing at the end where it just generates a simple path. We're going to go ahead and and get the optimal route. We want to find the best route in here. And so we've created a definition for the optimal route down here. Just scroll down for that. And we get the optimal route. uh we go ahead and put the information in including the Q table self uh start location in location next location route Q and it says while next location is not equal to end location. So while we can still go our start location equals self-loation to state start location. So we already have our best value for the start location. Uh the next state looks at the Q table and says, "Hey, what's uh the next one with the best value and then the next location, we go ahead and pull that in and we just append it." That's what's going on down here. And then our start location equals the next location. And we just go through all the steps. And we'll go ahead and run this. And now that we have our Q table, our um Q agent loaded, we're going to go ahead and uh take our Q agent, load them up with our alpha, gamma that we set up above um along with the location step, action, reward, state to location, and uh our goal is to plot a course between L9 and L1. And we're going to go through a 100 a thousand iterations on here. And so when I run that, it runs pretty quick. Uh why is this so fast? Um if you've been running neural networks and you've been doing all these other models, you sit here and wait a long time. Well, we're very small amount of data. These are all integers. These aren't float values. There's not a the math is not heavy on the on the processing end. And this is where Q tables are so powerful. If you have a small amount of information coming in, you very quickly uh get an answer off of this even though we went through it a thousand times to train it. And you'll see here we have L98 52 and one. And that's based on our reward table we had set up on there. And this is the shortest path going between these different uh setups in here. And if you remember on our reward table, uh you can see that if you start here, you can go to here. there's places you can't go. That's how this reward table was set up. So, I can only go to certain places. Uh so, kind of a little maze setup in there. And you can play with it. This is really fun uh setup to play with. Uh and you can see how you can take this whole code and you can like I was saying earlier, you can embed it into another setup and model and predictions where you put things into buckets and you're trying to guess the best investment, the best course of action. long as you can take that course of action and and uh uh reduce it down to a yes no um or if you're using text, you can use one hot encoder, which word is next. There's all kinds of things you can do with a Q table uh depending on just how much information you're putting in there. So, that wraps up our demo. In this demo, we've uh found the shortest distance between two paths based on whatever rules or state rewards we have to get from point A to point B and what available actions there are. Hello and welcome to this tutorial on deep learning. My name is Moan and in the next about one one and a half hours I will take you through what is deep learning and into TensorFlow environment to show you an example of deep learning. Now there are several applications of deep learning really very interesting and innovative applications and one of them is identifying the geographic location based on a picture. And how does this work? The way it works is pretty much we train an artificial neural network with millions of images which are tagged. their geoloccation is tagged and then when we feed a new picture, it will be able to identify the geoloccation of this new image. For example, you have all these images especially with maybe some significant monuments or or u significant locations and you train with millions of such images and then when you feed another image, it need not be exactly one of those that you have trained. It can be completely different. That is the whole idea of cleaning. It will be able to recognize for example that this is a picture from Paris because it is able to recognize the Eiffel Tower. So the way it works internally if we have to look a little bit under the hood is these images are nothing but this is digital information in the form of pixels. So each image could be a certain size. It can be 256 x 256 pixel kind of a resolution and then each pixel is either having a certain grade of color and all that is fed into the neural network and it then gets trained in and it's able to based on these pixels pixel information it is able to get trained and able to recognize the features and extract the features and thereby it is able to identif ify these images and the location of these images and then when you feed a new image it kind of based on the training it will be able to figure out where this images from. So that's the way a little bit under the hood how it works. So what are we going to do in this tutorial? We will see what is deep learning and what do we need for deep learning and one of the main components of deep learning is neural network. So we will see what is neural network what is a perceptron and how to implement logic gates like and or nor and so on using perceptrons the different types of neural networks and then applications of deep learning and we will also see how neural networks works. So how do we do the training of neural networks and at the end we will end up with a small demo code which will take you through in TensorFlow. Now in order to implement deep learning code there are multiple libraries or development environments that are available and TensorFlow is one of them. So the focus at the end of this would be on how to use TensorFlow to write a piece of code using Python as a programming language. And we will take up a an example which is a very common one which is like the hello world of deep learning the handwriting number recognition which is a mnest commonly known as mnest database. So we will take a look at mnest database and how we can train a neural network to recognize handwritten numbers. So that's what you will see in this particular video. So let's get started. What is deep learning? Deep learning is like a subset of what is known as a highlevel concept called artificial intelligence. You must be already familiar must have heard about this term artificial intelligence. So artificial intelligence is like the highle concept if you will and in order to implement artificial intelligence applications we use what is known as machine learning. And within machine learning, a subset of machine learning is deep learning. Machine learning is a little bit more generic concept. And deep learning is one type of machine learning if you will. And we will see a little later in maybe the following slides a little bit more in detail how deep learning is different from traditional machine learning. But to start with we can mention here that deep learning uses one of the differentiators between deep learning and traditional machine learning is that deep learning uses neural networks and we will talk about what are neural networks and how we can implement neural networks and so on and so forth as a part of this tutorial. So a little deeper into deep learning. Deep learning primarily involves working with complicated unstructured data compared to traditional machine learning where where we normally use structured data. In deep learning the data would be primarily images or voice or maybe text file. So and it is large amount of data as well and deep learning can handle complex operations. It involves complex operations. And the other difference between traditional machine learning and deep learning is that the feature extraction happens pretty much automatically. In traditional machine learning, feature engineering is done manually. The data scientists, we data scientists have to do feature engineering, feature extraction. But in deep learning that happens automatically and of course deep learning for large amounts of data, complicated unstructured data, deep learning gives very good performance. Now as I mentioned one of the secret sources of deep learning is neural networks. Let's see what neural networks is. Neural networks is based on our biological neurons. The whole concept of deep learning and artificial intelligence is based on human brain and human brain consists of billions of tiny stuff called neurons. And this is how a biological neuron looks. And this is how an artificial neuron looks. So neural networks is like a simulation of our human brain. Human brain has billions of biological neurons and we are trying to simulate the human brain using artificial neurons. This is how a biological neuron looks. It has dendrites and the corresponding component with an artificial neural network is or an artificial neuron are the inputs. They receive the inputs through dendrites and then there is the cell nucleus which is basically the processing unit in a way. So in artificial neuron also there is a piece which is an equivalent of this cell nucleus and based on the weights and biases. We will see what exactly weights and biases are as we move the input gets processed and that results in an output. In a biological neuron the output is sent through a synapse and in an artificial neuron there is an equivalent of that in the form of an output. And biological neurons are also interconnected. So there are billions of neurons which are interconnected. In the same way, artificial neurons are also interconnected. So this output of this neuron will be fed as an input to another neuron and so on. Now in neural network, one of the very basic units is a perceptron. So what is a perceptron? A perceptron can be considered as one of the fundamental units of neural networks. It can consist at least one neuron but sometimes it can be more than one neuron but you can create a perceptron with a single neuron and it can be used to perform certain functions. It can be used as a basic binary classifier. It can be trained to do some basic binary classification. And this is how a basic perceptron looks like. And this is nothing but a neuron. You have inputs x1, x2, x uh to xn and there is a summation function and then there is what is known as an activation function and based on this input what is known as the weighted sum. The activation function either gets gives an output like a zero or a one. So we say the neuron is either activated or not. So that's the way it works. So you get the inputs. These inputs are each of the inputs are multiplied by a weight and there is a bias that gets added and that whole thing is fed to an activation function and then that results in an output. And if the output is correct, it is accepted. It is wrong. If there is an error, then that error is fed back and the neuron then adjusts the weights and biases to give a new output and so on and so forth. So that's what is known as the training process of a neuron or a neural network. There's a concept called perceptron learning. So perceptron learning is again one of the very basic learning processes. The way it works is somewhat like this. So you have all these inputs like x1 to xn and each of these inputs is multiplied by a weight and then that sum this is the formula the equation. So that sum wi x i sigma of that which is a sum of all these product of x and w is added up and then a bias is added to that. The bias is not dependent on the input but or the input values but the bias is common for one neuron. However, the bias value keeps changing during the training process. Once the training is completed, the values of these weights W1, W2 and so on and the value of the bias gets fixed. So that is basically the whole training process and that is what is known as the perceptron training. So the weights and biases keep changing till you get the accurate output. And the summation is of course passed through the activation function. As you see here, this wixi summation plus b is passed through activation function and then the neuron gets either fired or not and based on that there will be an output that output is compared with the actual or expected value which is also known as labeled information. So this is the process of supervised learning. So the output is already known and um that is compared and thereby we know if there is an error or not and if there is an error the error is fed back and the weights and biases are updated accordingly till the error is reduced to the minimum. So this iterative process is known as perceptron learning or perceptron learning rule and this error needs to be minimized. So till the error is minimized this iteratively the weights and biases keep changing and that is what is the training process. So the whole idea is to update the weights and the bias of the perceptron till the error is minimized. The error need not be zero. The error may not ever reach zero. But the idea is to keep changing these weights and bias so that the error is minimum. the minimum possible that it can have. So this whole process is an iterative process and this is the iteration continues till either the error is zero which is uh unlikely situation or it is the minimum possible within these given conditions. Now in 1943 two scientists Warren Melikch and Walter Pittz came up with an experiment where they were able to implement the logical functions like and or and nor using neurons and that was a significant breakthrough in a sense. So they were able to come up with the most common logical gates. they were able to implement some of the most common logical gates which could take two inputs like a and b and then give a corresponding result. So for example in case of an and gate a and b and then the output is a b in case of an orgate it is a plus b and so on and so forth and they were able to do this using a single layer perceptron. Now most of these gates it was possible to use single layer perceptron except for XR and we will see why that is in a little bit. So this is how an AND gate works. The inputs A and B the output should be fired or the neuron should be fired only when both the inputs are one. So if you have 0 0 the output should be zero. For 0 1 it is again 0 1 0 again 0 and 1 1 the output should be one. So how do we implement this with a neuron? So it was found that by changing the values of weights it is possible to achieve this logic. So for example if we have equal weights like 7.7 and then if we take the sum of the weighted product. So for example 7 into 0 and then 7 into 0 will give you zero and so on and so forth. And in the last case when both the inputs are one you get a value which is greater than one which is the threshold. So only in this case the neuron gets activated and the output is there is an output. In all the other cases there is no output because the threshold value is one. So this is implementation of an AND gate using a single perceptron or a single neuron. Similarly an orgate in order to implement an orgate. In case of an orgate the output will be one if either of these inputs is one. So for example 01 will result in one or rather in all the cases it is one except for 0 0. So how do we implement this using a perceptron once again if you have a perceptron with weights for example 1.2. Now if you see here if in the first case when both are zero the output is zero. In the second case when it is 0 and 1.2 into 0 is 0 and then 1.2 2 into 1 is 1 and in the second case similarly the output is 1.2 in the last case when both the inputs are one the output is 2.4. So during the training process these weights will keep changing and then at one point where the weights are equal to w1 is equal to 1.2 and w2 is equal to 1.2 the system learns that it gives the correct output. So that is implementation of orgate using a single neuron or a single layer perceptron. Now XR gate this was one of the challenging ones. They tried to implement an XR gate with a single level perceptron but it was not possible and therefore in order to implement an XR. So this was like a a roadblock in the progress of U neural network. However, subsequently they realized that this can be implemented an XR gate can be implemented using a multi-level perceptron or MLP. So in this case there are two layers instead of a single layer. And this is how you can implement an XR gate. So you will see that X1 and X2 are the inputs and there is a hidden layer and that's why it is denoted as H3 and H4. And then you take the output of that and feed it to the output at 05 and provide a threshold here. So we will see here that this is a numerical calculation. So the weights are in this case for X1 it is 20 and minus 20 and once again 20 and minus 20. So these inputs are fed into H3 and H4. So you'll see here for H3 the input is 0 1 1 and for H4 it is 1 0 1 1 and if you now look at the output final output where the threshold is taken as one if you use a sigmoid with the threshold one you will see that in these two cases it is zero and in the last two cases it is one. So this is the implementation of XR. In case of XR only when one of the inputs is one you will get an output. So that is what we are seeing here. If we have either both the inputs are one or both the inputs are zero then the output should be zero. So that is what is an exclusive orgate. So it is exclusive because only one of the inputs should be one and then only you'll get an output of one which is satisfied by this condition. So this is a special implementation. An XR gate is a special implementation of a perceptron. Now that we got a good idea about perceptron, let's take a look at what is a neural network. So we have seen what is a perceptron, we have seen what is a neuron. So we will see what exactly is a neural network. So neural network is nothing but a network of these neurons and they are different types of neural networks. There are about five of them. These are artificial neural network, convolutional neural network, then recursive neural network or recurrent neural network, deep neural network and deep belief network. So, and each of these types of neural networks have a special you know they can solve a special kind of problems. For example, convolutional neural networks are very good at performing image processing and image recognition and so on. Whereas RNN are very good for speech recognition and also text analysis and so on. So each type has some special characteristics and they can uh they are good at performing certain special kind of tasks. What are some of the applications of deep learning? Deep learning is today used extensively in gaming. You must have heard about Alph Go which is a game created by a startup called Deep Mind which got acquired by Google and Alph Go is an AI which defeated the human world champion Lee Sadal in this game of go. So gaming is an area where deep learning is being extensively used and a lot of research happens in the area of gaming as well. In addition to that nowadays there are neural networks of special type called generative adversarial networks which can be used for synthesizing either images or music or text and so on and they can be used to compose music. So the neural network can be trained to compose a certain kind of music and autonomous cars. So you must be familiar with Google, Google's self-driving car and today a lot of automotive companies are investing in this space and uh deep learning is a core component of this autonomous cars. The cars are trained to recognize for example the road the the lane markings on the road signals any objects that are in front any obstruction and so on and so forth. So all this involves deep learning. So that's another major application and uh robots we have seen several robots including Sophia you may be familiar with Sophia who was given a citizenship by Saudi Arabia and there are several such robots which are very humanlike and the underlying technology in many of these robots is deep learning. Medical diagnostics and healthc care is another major area where deep learning is being used. And within healthcare diagnostics again there are multiple areas where deep learning and image recognition image processing can be used. For example for cancer detection as you may be aware if cancer is detected early on it can be cured. And one of the challenges is in the availability of specialists who can diagnose cancer using these diagnostic images and various scans and and so on and so forth. So the idea is to train neural network to perform some of these activities so that the load on the cancer specialist doctors or oncologists uh comes down and there is a lot of research happening here and there are already quite a few applications that are claimed to be performing better than human beings in this space. Can be lung cancer, it can be breast cancer and so on and so forth. So healthcare is a major area where deep learning is being applied. Let's take a look at the inner working of a neural network. So how does an artificial neural network let's say identify can we train a neural network to identify the shapes like squares and circles and triangles when these images are fed. So this is how it works. Any image is nothing but it is a digital information of the pixels. So in this particular case, let's say this is an image of 28x 28 pixel and this is an image of a square. There's a certain way in which the pixels are lit up. And so these pixels have a certain value maybe from 0 to 256 and 0 indicates that it is black or it is dark and 256 indicates it is completely it is white or lit up. So that is like an indication or a measure of the how the pixels are lit up. And so this is an image is let's say consisting of information of 784 pixels. So all the information what is inside this image can be kind of compressed into these 784 pixels. The way each of these pixels is lit up provides information about what exactly is the image. So we can train neural networks to use that information and identify the images. So let's take a look how this works. So each neuron the value if it is close to one that means it is white whereas if it is close to zero that means it is black. Now this is a an animation of how this whole thing works. So these pixels one of the ways of doing it is we can flatten this image and take this complete 784 pixels and feed that as input to our neural network. The neural network can consist of probably several layers. There can be a few hidden layers and then there is an input layer and an output layer. Now the input layer take these 784 pixels as input. The values of each of these pixels and then you get an output which can be of three types or three classes. One can be a square, a circle or a triangle. Now during the training process there will be initially obviously you feed this image and it will probably say it's a circle or it will say it's a triangle. So as a part of the training process, we then send that error back and the weights and the biases of these neurons are adjusted till it correctly identifies that this is a square. That is the whole training mechanism that happens out here. Now let's take a look at a circle. Same way. So you feed these 784 pixels. There is a certain pattern in which the pixels are lit up and the neural network is trained to identify that pattern and during the training process once again it would probably initially identify it incorrectly saying this is a square or a triangle and then that error is fed back and the weights and biases are adjusted finally till it finally gets the image correct. So that is the training process. So now we'll take a look at same way a triangle. So now if you feed another image which is consisting of triangles. So this is the training process. Now we have trained our neural network to classify these images into a triangle or a circle and a square. So now this neural network can identify these three types of objects. Now if you feed another image and it will be able to identify whether it's a square or a triangle or a circle. Now what is important to be observed is that when you feed a new image it is not necessary that the image or the the triangle is exactly in this position. Now the neural network actually identifies the patterns. So even if the triangle is let's say positioned here not exactly in the middle but maybe at the corner or in the side it would still identify that it is a triangle and that is the whole idea behind pattern recognition. So how does this training process work? This is a quick view of how the training process works. So we have seen that a neuron consists of inputs. It receives inputs and then there is a weighted sum which is nothing but this x i wi summation of that plus the bias and this is then fed to the activation function and that in turn gives us a output. Now during the training process initially obviously when you feed these images when you send maybe a square it will identify it as a triangle and when you maybe feed a triangle it will identify as a square and so on. So that error information is fed back and initially these weights can be random. Maybe all of them have zero values and then it will slowly keep changing. So the as a part of the training process the values of these weights W1, W2 up to WN keep changing in such a way that towards the end of the training process it should be able to identify these images correctly. So till then the weights are adjusted and that is known as the training process. So and these weights are numeric values. It could be 0.5.25.35 and so on. It could be positive or it could be negative. And the value that is coming here is the pixel value as we have seen. It can be anything between 0 to 1. You can scale it between 0 to 1 or 0 to 256 whichever way 0 being black and 256 being white. And then all the other colors in between. So that is the input. So these are numerical values. this multiplication or the product wi xi is a numerical value and the bias is also a numerical value. We need to keep in mind that the bias is fixed for a neuron. It doesn't change with the inputs whereas the weights are one per input. So that is one important point to be noted. So but the bias also keeps changing. Initially it will again have a random value but as a part of the training process the weights the values of the weights W1 W2 WN and the value of B will change and ultimately once the training process is complete these values are fixed for this particular neuron W1 W2 up to WN and plus the value of the B is also fixed for this particular neuron and in this way there will be multiple neurons and each there may be multiple levels of neurons here and that's the way the training process works. So this is another example of multilayer. So there are two hidden layers in between and then you have the input layer values coming from the input layer. And it goes through multiple layers, hidden layers. And then there is an output layer. And as you can see, there are weights and biases for each of these neurons in in each layer. And all of them gets keeps changing during the training process. And at the end of the training process, all these weights have a certain value. And that is a trained model. And those values will be fixed once the training is completed. All right. Then there is something known as activation function. Neural networks consists of one of the components in neural networks is activation function. And every neuron has an activation function. And there are different types of activation functions that are used. It could be a relu, it could be sigmoid and so on and so forth. And the activation function is what decides whether a neuron should be fired or not. So whether the output should be zero or one is decided by the activation function. And the activation function in turn takes the input which is the weighted sum. Remember we talked about wixi + b. That weighted sum is fed as a input to the activation function and then the output can be either a zero or a one. And there are different types of activation functions which are covered in an earlier video you might want to watch. All right. So as a part of the training process we feed the inputs the labeled data or the training data and then it gives an output which is the predicted output by the network which we indicate as yhat and then there is a labeled data because we for supervised learning we already know what should be the output. So that is the actual output and in the initial process before the training is complete obviously there will be error. So that is measured by what is known as a cost function. So the difference between the predicted output and the actual output is the error and the cost function can be defined in different ways. There are different types of cost functions. So in this case it is like the average of the squares of the error. So and then all the errors are added which can sometimes be called as sum of squares sum of square errors or SSC and that is then fed as a feedback in what is known as backward propagation or back propagation and that helps in the network adjusting the weights and biases and so the weights and biases get updated till this value the error value or the cost function is minimum. Now there is a optimization technique which is used here called gradient descent optimization and this algorithm works in a way that the error which is the cost function needs to be minimized. So there's a lot of mathematics that goes behind this. For example, they find the uh local minima and the global minima using the differentiation and so on and so forth. But the idea is this. So as a training process as the as the part of training the whole idea is to bring down the error which is like let's say this is the function the cost function at certain levels it is very high the cost value of the cost function the the output of the cost function is very high. So the weights have to be adjusted in such a way and also the bias of course that the cost function is minimized. So there is this optimization technique called gradient descent that is used and this is known as the learning rate. Now gradient descent you need to specify what should be the learning rate and the learning rate should be optimal because if you have a very high learning rate then the optimization will not converge because at some point it will cross over to the side. On the other hand, if you have very low learning rate, then it might take forever to convert. So you need to come up with the optimum value of the learning rate. And once that is done using the gradient descent optimization, the error function is reduced and that's like the end of the training process. All right. So this is another view of gradient descent. So this is how it looks. This is your cost function. the output of the cost function and that has to be minimized using gradient descent algorithm and these are like the parameters and weight could be one of them. So initially we start with certain random values so cost will be high and then the weights keep changing and in such a way that the cost function needs to come down and at some point it may reach the minimum value and then it may increase. So that is where the gradient descent algorithm decides that okay it has reached the minimum value and it will kind of try to stay here. This is known as the global minima. Now sometimes these curves may not be just for explanation purpose this has been drawn in a nice way but sometimes these curves can be pretty erratic. There can be some local minima here and then there is a peak and then and so on. So the whole idea of gradient descent optimization is to identify the global minima and to find the weights and the bias at that particular point. So that's what is gradient descent and then this is another example. So you can have these multiple local minima. So as you can see at this point when it is coming down it may appear like this is a minimum value but then it is not. This is actually the global minimum value and the gradient descent algorithm will make an effort to reach this level and not get stuck at this point. So the algorithm is already there and it knows how to identify this global minimum and that's what it does during the training process. Now in order to implement deep learning there are multiple platforms and languages that are available but the most common platform nowadays is tensorflow and so that's the reason we have uh this tutorial we have created this tutorial for tensorflow. So we will take you through a quick demo of how to write a tensorflow code using python and tensorflow is uh an opensource platform created by Google. So let's just take a look at the details of TensorFlow. And so this is a a library a Python library. So you can use Python or any other languages. It's also supported in other languages like Java and R and so on. But Python is the most common language that is used. So it is a library for developing deep learning applications especially using neural networks and it consists of primarily two parts if you will. So one is the tensors and then the other is the graphs or the flow. That's the way the name that's the reason for this kind of a name called tens of flow. So what are tensors? Tensors are like multi-dimensional arrays if you will. That's one way of looking at it. So usually you have a one-dimensional array. So first of all you can have what is known as a scalar which means a number. And then you have a one-dimensional array something like this which means this is like a set of numbers. So that is a one-dimensional array. Then you can have a two-dimensional array which is like a matrix and beyond that sometimes it gets difficult. So this is a three-dimensional array. But TensorFlow can handle many more dimensions. So it can have multi-dimensional arrays. That is the strength of TensorFlow and which makes computation deep learning computation much faster and that's the reason why TensorFlow is used for developing deep learning applications. So TensorFlow is a deep learning tool and this is the way it works. So the data basically flows in the form of tensors and the way the programming works as well is that you first create a graph of how to execute it and then you actually execute that particular graph in the form of what is known as a session. We will see this in the TensorFlow code as we move forward. So all the data is managed or manipulated in tensors and then the processing happens using this graphs. There are certain terms called like for example ranks of a tensor. The rank of a tensor is like a dimensional dimensionality in a way. So for example if it is scalar so there is just a number just one number the rank is supposed to be zero and then it can be a one-dimensional vector in which case the rank is supposed to be one and then you can have a two-dimensional vector typically like a matrix then in that case we say the rank is two and then if it is a three-dimensional array then it rank is three and so on. So it can have more than three as well. So it is possible that you can store multi-dimensional arrays in the form of tensors. So what are some of the properties of TensorFlow? I think today it is one of the most popular platform. TensorFlow is the most popular deep learning platform or library. It is open source. It's developed by Google, developed and maintained by Google. But it is open source. One of the most important things about TensorFlow is that it can run on CPUs as well as GPUs. GPU is a graphical processing unit just like CPU is central processing unit. Now in earlier days GPU was used for primarily for graphics and that's how the name has come and one of the reasons is that it cannot perform generic activities very efficiently like CPU but it can perform iterative actions or computations extremely fast and much faster than a CPU. So they are really good for computational activities and in deep learning there is a lot of iterative computation that happens. So in the form of matrix multiplication and so on. So GPUs are very well suited for this kind of computation and TensorFlow supports both GPU as well as CPU. And there's a certain way of writing code in TensorFlow. We will see as we go into the code. And of course, TensorFlow can be used for traditional machine learning as well, but then that would be an overkill. But just for understanding, it may be a good idea to start writing code for a normal machine learning use case so that you get a hang of how TensorFlow code works and then you can move into neural networks. So that is um just a suggestion but if you're already familiar with how TensorFlow works then probably yeah you can go straight into the neural networks part. So in this tutorial we will take the use case of recognizing handwritten digits. This is like a hello world of deep learning. And this is a nice little emnest database is a nice little database that has images of handwritten digits nicely formatted because very often in deep learning and neural networks. We end up spending a lot of time in preparing the data for training. And with MNEST database, we can avoid that. you already have the data in the right format which can be directly used for training and MNEST also offers a bunch of built-in utility functions that we can straight away use and call those functions without worrying about writing our own functions and that's one of the reasons why MNEST database is very popular for training purposes initially when people want to learn about deep learning and TensorFlow this is the database that is closed and it has a collection of 70,000 handwritten digits and a large part of them are for training. Then you have test just like in any machine learning process and then you have validation and all of them are labeled. So you have the images and their label and these images they look somewhat like this. So they are handwritten images collected from a lot of individuals. People have these are samples written by human beings. They have handwritten these numbers. These numbers going from 0 to 9. So people have written these numbers and then the images of those have been taken and formatted in such a way that it is very easy to handle. So that is MNEST database and the way we are going to implement this in our TensorFlow is we will feed this data especially the training data along with the label information and uh the data is basically these images are stored in the form of the pixel information as we have seen in one of the previous slides. All the images are nothing but these are pixels. So an image is nothing but an arrangement of pixels and the value of the pixel either it is lit up or it is not or in somewhere in between. That's how the images are stored and that is how they are fed into the neural network and for training. Once the network is trained when you provide a new image it will be able to identify within a certain error of course and for this we will use one of the simpler neural network configurations called softmax and for simplicity what we will do is we will flatten these pixels. So instead of taking them in a two-dimensional arrangement we just flatten them out. So for example, it starts from here. It is a 28x 28. So there are 784 pixels. So pixel number one starts here. It goes all the way up to 28. Then 29 starts here and goes up to 56 and so on. And the pixel number 784 is here. So we take all these pixels, flatten them out and feed them like one single line into our neural network. And this is a what is known as a softmax layer. What it does is once it is trained it will be able to identify what digit this is. So there are in this output layer there are 10 neurons each signifying a digit and at any given point of time when you feed an image only one of these 10 neurons gets activated. So for example, if this is trained properly and if you feed a number nine like this, then this particular neuron gets activated. So you get an output from this neuron. Let me just use uh a pen or a laser to show you here. Okay. So you're feeding a number nine. Let's say this has been trained. And now if you're feeding a number nine, this will get activated. Now let's say you feed one to the trained network then this neuron will get activated. If you feed two this neuron will get activated and so on. I hope you get the idea. So this is one type of a neural network or an activation function known as softmax layer. So that's what we will be using here. This is one of the simpler ones for quick and easy understanding. So this is how the code would look. We will go into our lab environment in the cloud and uh we will show you there directly but very quickly this is how the code looks and uh let me run you through briefly here and then we will go into the Jupyter notebook where the actual code is and we will run that as well. So as a first step first of all we are using Python here and that's why the syntax of the language is Python and the first step is to import the TensorFlow library. So and we do this by using this line of code saying import tensorflow as TF. TF is just for convenience. So you can name give any name and once you do this TF is TensorFlow is available as an object in the name of TF and then you can run its uh methods and accesses its attributes and so on and so forth and MNEST database is actually an integral part of TensorFlow and that's again another reason why we as a first step we always use this example MNEST database example. So you just simply import mnest database as well using this line of code and you slightly modify this so that the labels are in this format what is known as one hot true which means that the label information is stored like an array and uh let me just uh use the pen to show what exactly it is. So when you do this one hot true what happens is each label is stored in the form of an array of 10 digits and let's say the number is uh 8. Okay. So in this case all the remaining values there will be a bunch of zeros. So this is like array at position zero. This is at position one position two and so on and so forth. Let's say this is position 7. Then this is position 8 that will be one because our input is 8 and again position 9 will be zero. Okay. So one hot encoding this one hot encoding true will kind of load the data in such a way that the labels are in such a way that only one of the digits has a value of one and that indicates. So based on which digit is one we know what is the label. So in this case the eighth position is one. And therefore we know this sample data the value is 8. Similarly if you have a two here let's say then the labeled information will be somewhat like this. So you have your labels. So you have this as zero. The zeroth position the first position is also zero. The second position is one because this indicates number two. And then you have third as zero and so on. Okay. So that is the significance of this one hot true. All right. And then we can check how the data is uh looking by displaying the the data. And as I mentioned earlier, this is pretty much in the form of digital form like numbers. So all these are like pixel values. So you will not really see an image in this format. But there is a way to visualize that image. I will show you in a bit. And uh this tells you how many images are there in each set. So the training there are 55,000 images in training and in the test set there are 10,000 and then validation there are 5,000. So altogether there are 70,000 images. All right. So let's uh move on and we can view the actual image by uh using the mattplot clip library. And this is how you can view this is the code for viewing the images. And you can view them in color or you can view them in grayscale. So the cap is what tells in what way we want to view it. And what are the maximum values and the minimum values of the pixel values. So these are the max and minimum values. So of the pixel values. So maximum is one because this is a scaled value. So one means it is uh white and uh zero means it is black and in between is it can be anywhere in between black and white. And the way to train the model there is a certain way in which you write your TensorFlow code and um the first step is to create some placeholders and then you create a model. In this case, we will use the softmax model, one of the simplest ones. And um placeholders are primarily to get the data from outside into the neural network. So this is a very common mechanism that is used. And uh then of course you will have variables which are your remember these are your weights and biases. So for in our case there are 10 neurons and uh each neuron actually has 784 because each neuron takes all the inputs. So if we go back to our slide here actually every neuron takes all the 784 inputs right this is the first neuron it has it receives all the 784. This is the second neuron. This also receives all the 78. So each of these inputs needs to be multiplied with the weight. And that's what we are talking about here. So these are this is a a matrix of 784 values for each of the neurons. And uh so it is like a 10x 784 matrix because there are 10 neurons and uh similarly there are biases. Now remember I mentioned bias is only one per neuron. So it is not one per input unlike the weights. So therefore there are only 10 biases because there are only 10 neurons in this case. So that is what we are creating a variable for biases. So this is uh something little new in tensorflow you will see unlike our regular programming languages where everything is a variable. Here the variables can be of three different types. You have placeholders which are primarily used for feeding data. You have variables which can change during the course of computation. And then a third type which is not shown here are constants. So these are like fixed numbers. All right. So in a regular programming language you may have everything as variables or at the most variables and constants. But in TensorFlow you have three different types placeholders, variables and constants. And then you create what is known as a graph. So TensorFlow programming consists of graphs and tensors as I mentioned earlier. So this can be considered ultimately as a tensor and then the graph tells how to execute the whole implementation. So that the execution is stored in the form of a graph. And in this case what we are doing is we are doing a multiplication. TF you remember this TF was created as a TensorFlow object here. One more level one more. So TF is available here. Now, TensorFlow has what is known as a matrix multiplication or MACML function. So, that is what is being used here in this case. So, we are using the matrix multiplication of TensorFlow so that you multiply your input values X with W. Right? This is what we were doing. XW plus B. You're just adding B. And this is in very similar to one of the earlier slides where we saw sigma xi wi. So that's what we are doing here. Matrix multiplication is multiplying all the input values with the corresponding weights and then adding the bias. So that is the graph we created. And then we need to define what is our loss function and what is our optimizer. So in this case we again use the tensorflow's APIs. So tf.n NN softmax cross entropy with logits is the uh API that we will use and reduce mean is what is like the mechanism whereby which says that you reduce the error and optimizer for doing deduction of the error. What optimizer are we using? So we are using gradient descent optimizer. We discussed about this in couple of slides uh earlier. And for that you need to specify the learning rate. You remember we saw that there was a a slide somewhat like this and then you define what should be the learning rate. How fast you need to come down. That is the learning rate and this again needs to be tested and tried and to find out the optimum level of this learning rate. It shouldn't be very high in which case it will not converge or shouldn't be very low because it will in that case it will take very long. So you define the optimizer and then you call the method minimize for that optimizer and that will kickstart the training process and so far we've been creating the graph and in order to actually execute that graph we create what is known as a session and then we run that session and once the training is completed we specify how many times how many iterations we want it to run. So for example in this case we are saying thousand steps. So that is a exit strategy in a way. So you specify the exit condition. So a training will run for thousand iterations. And once that is done we can then evaluate the model using some of the techniques shown here. So let us get into the code quickly and see how it works. So this is our cloud environment. Now you can install TensorFlow on your local machine as well. I'm showing this demo on our existing cloud but you can also install TensorFlow on your local machine and uh there is a separate video on how to set up your TensorFlow environment. You can watch that if you want to install your local environment or you can go for other any cloud service like for example Google cloud, Amazon or cloud labs. Any of these you can use and u run and try the code. Okay. So it has got started. We will log. All right. So this is our deep learning tutorial uh code and uh this is our TensorFlow environment and uh so let's get started. The first we have seen a little bit of a code walk through uh in the slides as well. Now you will see the actual code in action. So the first thing we need to do is import tensorflow and then we will import the data and we need to adjust the data in such a way that the one hot is encoding is set to true one hot encoding right as I explained earlier. So in this case the label values will be shown appropriately and if we just check what is the type of the data. So you can see that this is a uh data sets Python data sets and if we check the number of images the way it looks. So this is how it looks it is an array of type float 32. Similarly the number if you want to see what is the number of training images there are 55,000 then there are test images 10,000 and then validation images 5,000. Now let's take a quick look at the data itself visualization. So we will use um mattplot lip for this. And um if we take a look at the shape now shape gives us like the dimension of the tensors or or or the arrays if you will. So in this case the training data set if we sees the size of the training data set using the method shape it says there are 55,000 and 55,000 by 784. So remember this 784 is nothing but the 28x 28 28 into 28. So that is equal to 784. So that's what it is uh showing. Now we can take just uh one image and just see what is the the first image and see what is the shape. So again size obviously it is only 784. Similarly you can look at the image itself the data of the first image itself. So this is how it it shows. So large part of it will probably be zeros because as you can imagine in the image only certain areas are written rest is uh blank. So that's why you will mostly see zeros either it is black or white but then there are these values are so the values are actually they are scaled so their values are between zero and one okay so this is what you're seeing so certain locations there are some values and then other locations there are zeros so that is how the data is stored and loaded if we want to actually see what is the value of the handwritten image If you want to view it, this is how you view it. So you create like do this reshape and um mattplot lib has this um feature to show you these images. So we will actually use the function called um im show and then if you pass this parameters appropriately you will be able to see the different images. Now I can change the values in this position. So which image we are looking at right? So we can say if I want to see what is there in maybe 5,000 right? So 5,000 has three similarly you can just say five what is in five five as 8 what is in 50 again eight. So basically, by the way, if you're wondering uh how I'm executing this code, shift enter. In case you're not familiar with Jupyter notebooks, shift enter is how you execute each cell, individual cell. And if you want to execute the entire program, you can go here and say run all. So that is how this code gets executed. And um here again we can check what is the maximum value and what is the minimum value of this pixel values. So as I mentioned this is it is scaled so therefore it is between the values lie between 1 and zero. Now this is where we create our model. The first thing is to create the required placeholders and variables and that's what we are doing here as we have seen in the slides. So we create one placeholder and we create two variables which is for the weights and biases. These two variables are actually matrices. So each variable has 784x 10 actual values. Okay. So one for this 10 is for each neuron. There are 10 neurons and 784 is for the pixel values inputs that are given which is 28 into 28. And the biases as I mentioned one for each neuron. So there will be 10 biases. they are stored in a variable by the name B. And this is the graph which is basically the multiplication of these matrix multiplication of X into W and then the bias is added for each of the neurons and the whole idea is to minimize the error. So let me just execute. I think this code is executed. Then we define what is our the Y- value is basically the label value. So this is another placeholder. We had x as one placeholder and y true as a second placeholder. And this will have values in the form of uh 10digit 10digit uh arrays. And uh since we said one hot encoded the position which has a one value indicates what is the label for that particular number. All right. Then we have cross entropy which is nothing but the loss loss function and we have the optimizer. We have chosen gradient descent as our optimizer. Then the training process itself. So the training process is nothing but to minimize the cross entropy which is again nothing but the loss function. So we define all of this in the form of a graph. So the up to here remember what we have done is we have not exactly executed any tensorflow code till now we are just preparing the graph the execution plan that's how the tensorflow code works. So the whole structure and format of this code will be completely different from how we normally do programming. So even with people with programming experience may find this a little difficult to understand it and it needs quite a bit of practice. So you may want to view this uh video also maybe a couple of times to understand this flow because the way TensorFlow programming is done is slightly different from the normal programming. Some of you who let's say have done uh maybe spark programming to some extent will be able to easily understand this. Uh but even in spark the the programming the code itself is pretty straightforward. Behind the scenes the execution happens slightly differently but in tensorflow even the code has to be written in a completely different way. So the code doesn't get executed uh in the same way as you have written. So that that's something you need to understand and a little bit of practice is needed for this. So so far what we have done up to here is creating the variables and feeding the variables and um or rather not feeding but setting up the variables and uh the graph that's all defining maybe the uh what kind of a network you want to use for example we want to use a softmax and so on. So you have created the variables how to load the data, loaded the data, viewed the data and prepared everything but you have not yet executed anything in TensorFlow. Now the next step is the execution in TensorFlow. So the first step for doing any execution in TensorFlow is to initialize the variables. So anytime you have any variables defined in your code, you have to run this piece of code always. So you need to basically create what is known as a a node for initializing. So this is a node. You still are not yet executing anything here. You just created a node for the initialization. So let us go ahead and create that. And here onwards is where you will actually execute your code uh in TensorFlow. And in order to execute the code, what you will need is a session. TensorFlow session. So tf session will give you a session. And there are a couple of different ways in which you can do this. But one of the most common methods of doing this is with what is known as a with loop. So you have a with tf. session as says and with a colon here. And this is like a block starting of the block and these indentations tell how far this block goes. And this session is valid till this block gets executed. So that is the purpose of creating this width block. This is known as a width block. So with tf do session as you say sess.run in it. Now ces.run run will execute a node that is specified here. So for example here we are saying ces run cess is basically an instance of the session right. So here we are saying tf dot session. So an instance of the session gets created and we are calling that sess and then we run a node within that one of the nodes in the graph. So one of the nodes here is in it. So we say run that particular node and that is when the initialization of the variables happens. Now what this does is if you have any variables in your code in our case we have W is a variable and B is a variable. So any variables that we created you have to run this code you have to run the initialization of these variables otherwise you will get an error. Okay so that is the that's what this is doing. Then we within this width block we specify a for loop and we are saying we want the system to iterate for thousand steps and perform the training. That's what this for loop does. Run training for thousand iterations. And what it is doing basically is it is fetching the data or these images. Remember there are about 50,000 images but it cannot get all the images in one shot because it will take up a lot of memory and performance issues will be there. So this is a very common way of performing deep learning training. You always do in batches. So we have maybe 50,000 images but you always do it in batches of 100 or maybe 500 depending on the size of your system and so on and so forth. So in this case we are saying okay get me 100 uh images at a time and get me only the training images. Remember we use only the training data for training purpose and then we use test data for test purpose. You must be familiar with machine learning. So you must be aware of this but in case you are not in machine learning also not this is not specific to deep learning but in machine learning in general you have what is known as training data set and test data set. your available data typically you will be splitting into two parts and using the training data set for training purpose and then to see how well the model has been trained you use the test data set to check or test the validity or the accuracy of the model. So that's what we are doing here and you observe here that we are actually calling an MNEST function here. So we are saying mnest train next batch right? So this is the advantage of using MNEST database because they have provided some very nice helper functions which are readily available otherwise this activity itself we would have had to write a piece of code to fetch this data in batches that itself is a a lengthy exercise. So we can avoid all that if we are using MNEST database and that's why we use this for the initial learning phase. Okay. So when we say fetch what it will do is it will fetch the images into X and the labels into Y and then you use this batch of 100 images and you run the training. So ces.run basically what we are doing here is we are running the training mechanism which is nothing but it passes this through the neural network passes the images through the neural network finds out what is the output and if the output obviously the initially it will be wrong. So all that feedback is given back to the neural network and thereby all the W's and B's get updated till it reaches thousand iterations. In this case the exit criteria is thousand but you can also specify probably accuracy rate or something like that for the as an exit criteria. So here it is it it just says that okay this particular image was wrongly predicted so you need to update your weights and biases. That's the feedback given to each neuron and that is run for thousand iterations and typically by the end of this thousand iterations the model would have learned to recognize these handwritten images. Obviously it will not be 100% accurate. Okay. So once that is done after so this happens for thousand iterations. Once that is done, you then test the accuracy of these models by using the test data set. Right? So this is what we are trying to do here. The code may appear a little complicated because if you're seeing this for the first time, you need to understand uh the various methods of TensorFlow and so on. But it is basically comparing the output with what has been what is actually there. That's all it is doing. So you have your test data and uh you're trying to find out what is the actual value and what is the predicted value and seeing whether they are equal or not tf.equal right and how many of them are correct and so on and so forth and based on that the accuracy is uh calculated as well. So this is the accuracy and uh that is what we are trying to see how accurate the model is in predicting these uh numbers or these digits. Okay. So let us run this. This entire thing is in one cell. So we will have to just run it in one shot. It may take a little while. Let us see. And uh not bad. So it has finished the thousand iterations. And what we see here as an output is the accuracy. So we see that the accuracy of this model is around 91%. Okay. Now, which is pretty good for such a short exercise. Within such a short time, we got 90% accuracy. However, in real life, this is probably not sufficient. So, there are other ways in to increase the accuracy. We will see it probably in some of the later tutorials how to improve this accuracy, how to change maybe the hyperparameters like number of neurons or number of layers and so on and so forth. and uh so that this accuracy can be increased beyond 90%. Hello and welcome to the TensorFlow object detection API tutorial. In this video, I will walk you through the TensorFlow code to perform object detection in a video. So let's get started. This part is basically you're importing all the libraries. We need a lot of these libraries. for example, numpy, we need image io, datetime and pill and so on and so forth and of course math plot lib. So we import all these libraries and then there are a bunch of variables which have some paths for the files and folders. So this is regular stuff. Let's keep moving. Then we import the mattplot lib and make it inline and uh a few more imports. All right. And then these are some warnings. So we can just ignore them. So if I run this code once again, it will go away. All right. And then here onwards we do the model preparation. Now what we're going to do is we're going to use an existing neural network model. So we are not going to train a new one because that really will uh take a long time and uh it needs a lot of uh computation resources and so on and it is really not required. There are already models that have been trained and in this case it is the SSD with mobile net. That's the model that we are going to use and uh this model is trained to detect objects and uh it is readily available as open source. So we can actually use this and if you want to use other models there are a few more models available. So you can click on this link here and uh let me just take you there. There are a few more models but we have chosen this particular one because this is uh faster. It may not be very accurate but that is one of the faster models. But on this link you will see a lot of other models that are readily available. These are trained models. Some of them would take a little longer but they may be more accurate and so on. So you can probably play around with these other models. Okay. So we will be using that model. So this piece of code this line is basically importing that model. And this is also known as a frozen model. The term we use is frozen model. So we import download and import that and then we will actually use that model in our code. All right. So these two cells we have downloaded and import the model and then once it is available locally we will then load this into our program. All right. So we are loading this into memory and uh you need to perform a couple of additional steps which is basically we need to to map the numbers to text. As you may be aware when we actually build the model and when we run predictions the model will not give a text the output of the model is usually a number. So we need to map that to a text. So for example, if the network predicts that the output is five, we know that five means it is an aeroplane. So things like that. So this mapping is done in this next cell. All right. So let's keep moving. And then we have a helper code which will basically load the data or load the images and transform into numpy arrays. This is also used in doing object detection in images. So we are actually going to reuse because video is nothing but it consists of frames which in turn are images. So we are going to pretty much use reuse the same code which we used for doing object detection in images. So this is where the actual detection starts. So here this is the path for where the images are stored. So this is here once again we are reusing the code which we wrote for detecting objects in an image. So this is the path where the images were stored and this is the extension and this was done for about two or three images. So we will continue to use this and uh we go down I'll skip this section. So this is the cell where we are actually loading the video and converting it into frames and then using frame by frame we are detecting the objects in the image. So in this code what we are doing basically is there a few lines of code what they do is basically once they find an object a box will be drawn around those uh each of those objects and the input file the name of the input video file is uh traffic it is the extension is mp4 and uh we have this video reader it's a excellent object which is basically part of this class called image IO so we can read and write videos using that and uh the video that we are going to use is traffic.mpp4. You can use any MP4 file but in our case I picked up video which has uh like car. So let me just show you. So this is in this object detection folder. I have this MP4 file. I'll just quickly play this video. It's a little slow. Yeah. Okay. So here we go. This is the video. It's a short one, relatively small video. So that for this particular demo and what it will do is once we run our code it will detect each of these cars and it will annotate them as cars. So in this particular video we only have cars. We can later on see with another video. I think I have cat here. So we can also try with that. But let's first check with this uh traffic video. So let me go back. So we will be reading this frame by frame and um no actually we will be reading the video file but then we will be analyzing it frame by frame and we will be reading them at 10 frames per second that is the rate we are mentioning here and analyzing it and then annotating and then writing it back. So you will see that we will have a video file named something like this traffic annotated and um we will see the annotated video. So let's go back and run through this piece of code and then we will come back and see the annotated uh video. This might take a little while. So I will pause the video after running this particular cell and then come back to show you the results. All right. So let's go ahead and run it. So it is running now. And it is also important that at the end you close the video writer so that it is similar to a file pointer. When you open a file you should also make sure you close it so that it doesn't hog the resources. So it's very similar at the end of it the last piece or last line of code should be video_riter.close. All right. So I'll pause and then I'll come back. Okay. So I will see you in a little bit. All right. So now as you can see here the processing is done. The hourglass has disappeared. That means the video has been processed. So let's go back and check the annotated video. We'll go back to my file manager. So this was the original traffic.mpp4. And now you have here traffic_an annotatedmpp4. So let's go and run this and see how it looks. You see here got each of these cars are getting detected. Let me pause and show you. So we pause here. It says car 70%. Let us allow it to go a little further. It detects something on top. What is that truck? Okay. So I think because of the board on top it somehow thinks there is a truck. Let's lay some more and see if it detects anything else. So this is again a car looks like. So let us Yeah. So this is a car and it has confidence level of 69%. Okay. This is again a car. All right. So basically till the end it goes and detects each and every car that is passing by. Now we can quickly repeat this process for another video. Let me just show you the other video which is a cat. Again there is uh this cat is not really moving or anything but it is just standing there staring and moving a little slowly and uh our application will our network will detect that this is a cat and uh even when the cat moves a little bit in the other direction it'll continue to detect and show that it is a cat. Okay. So yeah so this is how the original video is. Let's go ahead and change our code to analyze this one and see if it detects our network detects this cat. Close this. Here we go. And I'll go back to my code. All we need to do is change this traffic to cat. The extension it will automatically pick up because it is given here and then it will run through. So very quickly once again what it is doing is this video reader video reader has a a neat little feature or interface whereby you can say for frame in video reader. So it will basically provide frame by frame. So you're in a loop frame by frame and then you take that each frame that is given to you. You take it and analyze it as if it is an image individual image. So that's the way it works. So it is uh very easy to handle this. All right. So now let's once again run just this cell. The rest of the stuff remains the same. So I will run this cell again. It will take a little while. So the hourglasses come back. I will pause and then come back in a little while. All right. So the processing is done. Let's go and check the annotated video. Go here. So we have cat annotated.mpp4. Let's play this. All right. So you can see here it is detecting the cat. And in the beginning you also saw it detected something else here. There looks like it detected one more object. So let's just go back and see what it has detected yet. Let's see. Yes. So what is it trying to show here? It's too small. Not able to see. But uh it is trying to detect something. I think it is saying it is a car. I don't know. All right. Okay. So in this video there's only pretty much only one object which is a cat. And uh let's wait for some time and see if it continues to detect it when the cat turns around and moves as well. Just in a little bit that's going to happen and we will see. There we go. And in spite of turning the other way, I think our network is able to detect that it is a cat. So let me freeze and then show here. It is actually still continues to detect it as a cat. All right. So that's pretty much it. I think that's the only object that it detects in this particular video. Okay, so close this. So that's pretty much it. Thank you very much for watching this video and you have a great day and in case you have any questions, please uh put them below the video here and we will be more than happy to get back to you and make sure you put your email id so that we can contact you in case you have any questions. Thank you once again. Bye-bye. >> Today we're going to be covering the convolutional neural network tutorial. Do you know how deep learning recognizes the objects in an image? And really this particular neural network is how image recognition works. It's very central, one of the biggest building blocks for image recognition. It does it using convolution neural network. And we over here we have the basic picture of a hummingbird. Pixels of an image fed as input. You have your input layer coming in. So it takes that graphic and puts it into the input layer. You have all your hidden layers and then you have your output layer and your output layer. one of those is going to light up and say, "Oh, it's a bird." We're going to go into depth. We're going to actually go back and forth on this a number of times today. So, if you're not catching all the image, um, don't worry. We're going to get into the details. So, we have our input layer accepts the pixels of the image as input in the form of arrays. And you can see up here where they've actually, um, labeled each block of the bird in different arrays. So, we'll dive into deep as to how that looks like and how those matrixes are set up. your hidden layer carry out feature extraction by performing certain calculations and manipulation. So this is the part that kind of reorganizes that picture multiple ways until we get some data that's easy to read for the neural network. This layer uses a matrix filter and performs convolution operation to detect patterns in the image. And if you remember that convolution means to coil or to twist. So we're going to twist the data around and alter it and use that operation to detect a new pattern. There are multiple hidden layers like convolution layer relu is how that is pronounced and that's the rectified linear unit that has to do with the activation function that's used. Pooling layer also uses multiple filters to detect edges, corners, eyes, feathers, beak, etc. And just like the term says, pooling is pooling information together. And we'll look into that a lot closer here. So if you're if it's a little confusing now, we'll dig in deep and try to get you uh squared away with that. And then finally, there is a fully connected layer that identifies the object in the image. So we have these different layers coming through in the hidden layers and they come into the final area and that's where we have say one node or one neural network entity that lights up that says it's a bird. What's in it for you? We're going to cover an introduction to the CNN. What is convolution neural network? how CNN recognizes images. We're going to dig deeper into that and really look at the individual layers in the convolutional neural network. And finally, we do a use case implementation using the CNN. We'll begin our introduction to the CNN by introducing the pioneer of convolutional neural network, Yan Leon. He was the director of Facebook AI research group. Built the first convolutional neural network called Lynette in 1988. So these have been around for a while and have had a chance to mature over the years. It was used for character recognition tasks like reading zip code digits. Imagine processing mail and automating that process. CNN is a feed forward neural network that is generally used to analyze visual images by producing data with a grid-like topology. A CNN is also known as a convenet. And very key to this is we are looking at images. That was what this was designed for. And you'll see the different layers as we dig in mirror some of the other some of them are actually now used since we're using uh TensorFlow and KAS in our code later on. You'll see that some of those layers appear in a lot of your other neural network frameworks. Uh but in this case, this is very central to processing images and doing so in a variety that captures multiple images and really drills down into their different features. In this example here, you see flowers of two varieties, orchid and a rose. I think the orchid is much more dainty and beautiful and the rose smells quite beautiful. I have a couple rose bushes in my yard. Uh they go into the input layer. That data is then sent to all the different nodes in the next layer, one of the hidden layers based on its different weights and its setup. It then comes out and gives those a new value. Those values then are uh multiplied by their weights and go to the next hidden layer and so on. And then you have the output layer and one of those nodes comes out and says it's an orchid and the other one comes out and says it's a rose depending on how was well it was trained. What separates the CNN or the convolutional neural network from other neural networks is a convolutional operation forms a basis of any convolutional neural network. In a CNN, every image is represented in the form of arrays of pixel values. So here we have a real image of the digit 8 uh that then gets put onto its pixel values represented in the form of an array. In this case you have a two-dimensional array. And then you can see in the final in form we transform the digit 8 into its representational form of pixels of zeros and one where the ones represent in this case the black part of the eight and the zeros represent the white background. To understand the convolution neural network or how that convolutional operation works, we're going to take a side step and look at matrices. In this case, we're going to simplify it. We're going to take two matrices A and B of one dimension. Now kind of separate this from your thinking as we learn that you want to focus just on the matrix aspect of this and then we'll bring that back together and see what that looks like when we put the pieces for the convolutional operation. Here we've set up two arrays. We have uh in this case they're a single dimension matrix and we have a= 537597 and we have b= 1 2 3. So in the convolution as it comes in there it's going to look at these two and we're going to start by doing multiplying them. a * b. And so we multiply the arrays elementwise and we get 5 6 where five is the 5 * 1, 6 is 3 * 2, and then the other six is 2 * 3. And since the two arrays aren't the same size, they're not the same setup. We're going to just truncate the first one, and we're going to look at the second array multiplied just by the first three elements of the first array. Now, that's going to be a little confusing. Remember, a computer gets to repeat these processes hundreds of times. So, we're not going to just forget those other numbers later on. We'll see. We'll bring those back in. And then we have the sum of the product. In this case, 5 + 6 + 6= 17. So, in our A * B, our very first digit in that matrix of A * B is 17. And if you remember, I said we're not going to forget the other digits. So, we now have 325. We move one set over and we take 325 and we multiply that times B. And you'll see that 3 * 1 is 3, 2 * 2 is 4, and so on and so on. We sum it up. So now we have the second digit of our A * B product in the matrix. And we continue on with that same thing. So on and so on. So then we would go from uh 375 to 759 to 597. This short matrix that that we have for A, we've now covered all the different entities in A that match three different levels of B. Now, in a little bit, we're going to cover where we use this math at this multiplying of matrices and how that works. Uh, but it's important to understand that we're going through the matrix and multiplying the different parts to it to match the smaller matrix with the larger matrix. I know a lot of people get lost at is, you know, what's going on here with these matrixes. Uh, scary math. Not really that scary when you break it down. We're looking at a section of A and we're comparing it to B. So when you break that down in your mind like that, you realize, okay, so I'm I'm just taking these two matrices and comparing them and I'm bringing the value down into one matrix A * B. We're deucing that information in a way that will help the computer see different aspects. Let's go ahead and flip over again back to our images. Here we are back to our images talking about going to the most basic two-dimensional image you can get to. Consider the following two images. the image for the symbol backslash. When you press the backslash, the above image is processed. And you can see there for the image for the forward slash is the opposite. So we click the forward slash button that flips. Uh very basic. We have four pixels going in. Can't get any more basic than that. Here we have a little bit more complicated picture. We take a real image of a smiley face. U then we represent that in the form of black and white pixels. So if this was an image in the computer, it's black and white. And like we saw before, we convert this into the zeros and ones. So where the other one would have just been a matrix of just four dots, now we have a significantly larger image coming in. So don't worry, we're going to bring this all together here in just a little bit. Layers in convolutional neural network. When we're looking at this, we have our convolution layer. And that really is the central aspect of processing images and the convolutional neural network. That's why we have it. And then that's going to be feeding in. And you have your relu layer which is you know as we talked about the rectified linear unit. We'll talk about that a little bit later. The relu is an how it act is how that layer is activated is the math behind it. What makes the neurons fire you'll see that in a lot of other neural networks when you're using it just by itself. It's for processing smaller amounts of data where you use the atom activation feature for large data coming in. Now, because we're processing small amounts of data in each image, the relu layer works great. You have your pooling layer. That's where you're pulling the data together. Pooling is a neural network term. It's very commonly used. I like to use a term reduce. So, if you're coming from the map and reduce side, you'll see that we're mapping all this data through all these networks and then we're going to reduce it. We're going to pull it together. And then finally, we have the fully connected layer. That's where our output's going to come out. So we have started to look at matrixes. We've started to look at the convolutional layer and where it fits in and everything. We've taken a look at images. So we're going to focus more on the convolution layer since this is a convolutional neural network. A convolution layer has a number of filters and perform convolution operation. Every image is considered as a matrix of pixel values. Consider the following 5x5 image whose pixel values are only 0 and one. Now, obviously, when we're dealing with color, there's all kinds of things that come in on color processing, but we want to keep it simple and just keep it black and white. And so, we have our image pixels. Uh, so we're sliding the filter matrix over the image and computing the dot product to detect the patterns. And right here, you're going to ask where does this filter come from? This is a bit confusing because the filter is going to be derived uh later on. We build the filters when we program or train our model. So you don't need to worry what the filter actually is. What you do need to understand how a convolution layer works is what is the filter doing filter. And you'll have many filters. You don't have just one filter. You'll have lots of filters that are going to look for different aspects. And so the filter might be looking for just edges. It might be looking for different parts. We'll cover that a little bit more detail in a minute. Right now we're just focusing on how the filter works as a matrix. Remember earlier we talked about multiplying matrices together. And here we have our two-dimensional matrix. And you can see we take the filter and we multiply it in the upper left image. And you can see right here 1 * 1, 1 * 0, 1 * 1. We multiply those all together, then sum them. And we end up with a convolved feature of four. We're going to take that and sliding the filter matrix over the image and computing the dotproduct to detect patterns. So we're just going to slide this over. We're going to predict the first one and slide it over one notch. Predict the second one and so on and so on all the way through until we have a new matrix. And this matrix which is the same size as a filter has reduced the image and whatever filter whatever that's filtering out is going to be looking at just those features reduced down to a smaller uh matrix. So once the feature maps are extracted the next step is to move them to the relu layer. So the relu layer the next step first is going to perform an elementwise operation. So each of those maps coming in if there's negative pixels, so it sets all the negative pixels to zero. Um, and you can see this nice graph where it just zeros out the negatives and then you have a value that goes from zero up to whatever value is coming out of the matrix. This introduces nonlinearity to the network. Uh, so up until now we have a when we say linearity, we're talking about the fact that the feature has a value. So it's a linear feature. this feature um came up and has let's say the feature is the edge of the beak. You know, it's like or that backslash that we saw. Um it'll look at that and say, "Okay, this feature has a value from -10 to 10 in this case." Um if it was one, it' say, "Yeah, this might be a beak. It might not. Might be an edge right there." A minus 5 means no, we're not even going to look at it. It's a zero. And so we end up with an output. And the output takes all these feature all these filtered features. Remember, we're not just running one filter on this. We're running a number of filters on this image and so we end up with an rectified feature map that is looking at just the features coming through and how they weigh in from our filters. So here we have an input of a it looks like a toucan bird very exotic looking real image is scanned in multiple convolution and the relu layers for locating features and you can see up here is turn it into a black and white image and in this case we're looking in the upper right hand corner for a feature and that box scans over a lot of times it doesn't scan one pixel at a time a lot of times it will skip by two or three or four pixels uh to speed up the process that's one of the ways you can compens compensate if you don't have enough resources on your computation for large images and it's not just one filter slowly goes across the image. Uh you have multiple filters have been programmed in there. So you're looking at a lot of different filters going over the different aspects of the image and just sliding across there and forming a new matrix. One more aspect to note about the relu layer is we're not just having one relu coming in. Uh so not only do we have multiple features going through but we're generating multiple relu layers for locating the features. That's very important to note you know so we have a quite a bundle. We have multiple filters multiple railu uh which brings us to the next step forward propagation. Now we're going to look at the pooling layer. The rectified feature map now goes through a pooling layer. Pooling is a downsampling operation that reduces the dimensionality of the feature map. That's all we're trying to do. We're trying to take a huge amount of information and reduce it down to a single answer. This is a specific kind of bird. This is an iris. This is a rose. So you have a rectified feature map. And you see here we have a rectified feature map coming in. Um we set the max pooling with a 2x two filters and a stride of two. And if you remember correctly, I talked about not going one pixel at a time. Uh well, that's where the stride comes in. we end up with a 2x two pulled feature map. But instead of moving one over each time and looking at every possible combination, we skip a we skip a few there. We go by two. We skip every other pixel and we just do every other one. Um and this reduces our rectified feature map which is you can see over here 16x 16 to a 4x4. So we're continually trying to filter and reduce our data so that we can get to something we can manage. And over here you see that we have the max uh 3 4 1 and 2 and in the max pooling we're looking for the max value a little bit different than what we were looking at before. So coming from the rectified feature we're now finding the max value and then we're pooling those features together. So instead of think of this as image of the map, think of this as how valuable is a feature in that area. How much of a feature value do we have? And we just want to find the best or the maximum feature for that area. They might have that one piece of the filter of the beak said, "Oh, I see a one in this beak in this image." And then it skips over and says, "I see a three in this image." And says, "Oh, this one is rated as a four." We don't want to sum it together cuz then, you know, you might have like five ones and it'll say, "Ah, five." But you might have uh four zeros and one 10. And that 10 says, "Well, this is definitely a beak." Where the ones will say, "Probably not a beak." A little strange analogy since we're looking at a bird, but you can see how that pulled feature map comes down and we're just looking for the max value in each one of those matrixes. Pooling layer uses different filters to identify different parts of the image like edges, corners, body, feathers, eyes, beak, etc. Um, I know I focus mainly on the beak, but obviously uh each feature could be each a different part of the bird coming in. So, let's take a look at what that looks like. structure of a convolution neural network. So far, this is where we're at right now. We have our input image coming in. And then we use our filters and there's multiple filters on there that are being developed to kind of twist and change that data. And so we multiply the matrixes. We take that little filter, maybe it's a 2x two, we multiply it by each piece of the image. And if we step two, then it's every other piece of the image that generates multiple convolution layers. So we have a number of convolution layers. we have um set up in there is looking at that data. We then take those convolution layers, we run them through the relu setup and then once we've done through the relu setup and we have multiple relu going on multiple layers that are relu then we're going to take those multiple layers and we're going to be pooling them. So now we have the pooling layers or multiple poolings going on. Up until this point we're dealing with u sometimes it's multiple dimensions. So you can have three dimensions. Some strange data setups that aren't doing images but looking at other things, they can have four, five, six, seven dimensions. Uh so right now we're looking at 2D image dimensions coming in into the pooling layer. So the next step is we want to reduce those dimensions or flatten them. So flattening flattening is a process of converting all of the resultant two-dimensional arrays from pulled feature map into a single long continuous linear vector. So over here you see where we have a pulled feature map. Maybe that's the bird wing and it has values 6847 and we want to just flatten this out and turn it into 6847 or a single linear vector. And we find out that not only do we do each of the pulled feature maps, we do all of them into one long linear vector. So now we've gone through our convolutional neural network part and we have the input layer into the next setup. All we've done is taken all those different pooling layers and we've flattened them out and combined them into a single linear vector going in. So after we've done the flattening, we have just a quick recap because we've covered so much. So it's important to go back and take a look at each of the steps. We've gone through the structure of the network so far is we have our convolution where we twist it and we filter it and multiply the matrices. We end up with our convolutional layer which uses the relu to figure out the values going out into the pooling. as you have numerous convolution layers that then create numerous pooling layers pooling that data together which is the max value which one we want to send forward we want to send the best value and then we're going to take all of that from each of the pooling layers and we're going to flatten it and we're going to combine them into a single input going into the final layer. Once you get to that step you might be looking at that going boy that looks like the normal input to most neural network and you're correct it is. So once we have the flattened matrix from the pooling layer that becomes our input. So the pooling layer is fed as an input to the fully connected layer to classify the image. And so you can see as our flattened matrix comes in in this case we have the pixels from the flattened matrix fed as an input back to our toucan or whatever that kind of bird that is. Um I need one of these to identify which kind of bird that is. It comes into our forward propagation network. Uh, and that will then have the different weights coming down across. And then finally, it selects that that's a bird and that it's not a dog or a cat in this case, even though it's not labeled. The final layer there in red is our output layer. Our final output layer that says bird, cat, or dog. So, quick recap of everything we've covered so far. We have our input image, which is twisted and multiply. The filters are multiplied times the uh matri the two matrixes multiplied all the filters to create our convolution layer. Our convolution layers there's multiple layers in there because it's all building multiple layers off the different filters. Then goes through the reu as this activation and that creates our pooling. And so once we get into the pooling layer, we then the pooling look for who's the best. What's the max value coming in from our convolution? And then we take that layer and we flatten it and then it goes into a fully connected layer. our fully connected neural network and then to the output. And here we can see the entire process how the CNN recognizes a bird. This is kind of nice because it's showing the little pixels and where they're going. You can see the filter is generating this convolution network and that filter shows up in the bottom part of the convolution network. And then based on that, it uses the relu for the pooling. The pooling then find out which one's the best and so on all the way to the fully connected layer at the end or the classification in the output layer. So that'd be a classification neural network at the end. So we covered a lot of theory up till now and you can imagine each one of these steps has to be broken down in code. So putting that together can be a little complicated. Not that each step of the process is overly complicated, but because we have so many steps. Uh we have 1 2 3 4 five different steps going on here with substeps in there. We're going to break that down and walk through that in code. So in our use case implementation using the CNN, we'll be using the CR10 data set from Canadian Institute for Advanced Research for classifying images across 10 categories. Unfortunately, they don't let me know whether it's going to be a toucan or some other kind of bird. But we do get to find out whether it can categorize between a ship, a frog, deer, bird, airplane, automobile, cat, dog, horse, truck. So that's a lot of fun. And then if you're looking anything in the news at all of our automated cars and everything else, you can see where this kind of processing is so important in today's world and very cutting edge. As far as what's coming out in the commercial deployment, I mean, this is really cool stuff. We're starting to see this just about everywhere in industry. Uh so great time to be playing with this and figuring it all out. Let's go ahead and dive into the code and see what that looks like when we're actually writing our script. Before we go on, let's do uh one more quick look at what we have here. Let's just take a look at data batch one keys. And remember in Jupyter notebook, I can get by with not doing the print statement. If I put a variable down there, it'll just display the variable. And you can see under data batch one for the keys, since this is a dictionary, we have the batch one label, data, and file names. Uh so you can actually see how it's broken up in our data set. So for the next step or step four as we're calling it uh we want to display the images using mattplot library. There's many ways to display the images. You could even uh well there's other ways to drill into it but mattplot library is really good for this. And we'll also look at our first reshape uh setup or shaping the data so you can have a little glimpse into what that means. Uh so we're going to start by importing our map plot. And of course since I am doing Jupyter notebook I need to do the mapplot inline command so it shows up on my page. So here we go. We're going to import mapplot library.pipplot is plt. And if you remember mapplot library the pipplot is like a canvas that we paint stuff onto. And there's my percentage sign mapplot library in line. So it's going to show up in my notebook. And then of course we're going to import numpy as np for our numbers python array setup. And let's go ahead and set x equals to data batch one. So this will pull in all the data going into the x value. And then because this is just a long stream of binary data, uh we need to go a little bit of reshaping. So in here we have to go ahead and reshape the data. We have 10,000 images. Okay, that looks correct. And this is kind of an interesting thing. It took me a little bit to I had to go research this myself to figure out what's going on with this data. And what it is is it's a 32x 32 picture. And let me do this. Let me go ahead and do a drawing pad on here. Uh, so we have 32 bits by 32 bits and it's in color. So there's three bits of color. Now, I don't know why the data is particularly like this. It probably has to do with how they originally encoded it. But most pictures put the three afterward. So what we're doing here is we're going to take uh the shape. We're going to take the data, which is just a long stream of information, and we're going to break it up into 10,000 pieces. And those 10,000 pieces then are broken into three pieces each. And those three pieces then are 32x 32. You can look at this like an oldfashioned projector where they have the red screen or the red projector, the blue projector and the green projector and they add them all together and each one of those is a 32x 32bit. So that's probably how this was originally formatted was in that kind of ideal. Things have changed. So we're going to transpose it and we're going to take the three which was here and we're going to put it at the end. So the first part is reshaping the data from a single line of bit data or whatever format it is into 10,000 by3x 32 by 32. And then we're going to transpose the color factor to the last place. So it's the image then the 32x 32 in the middle. That's this part right here. And then finally we're going to take this uh which is three bits of data and put it at the end. So it's more like we do we process images now. And then as type, this is really important that we're going to use an integer 8. You can come in here and you'll see a lot of these they'll try to do this with a float or a float 64. What you got to remember though is a float uses a lot of memory. So once you switch this into uh something that's not integer 8, which is goes up to 128, you are just going to the the amount of RAM, let me just put that in here, is going to go way up. the amount of RAM that it loads. Uh so you want to go ahead and use this. You can try the other ones and see what happens if you have a lot of RAM on your computer, but for this exercise, this will work just fine. And let's go ahead and take that and run this. So now our X variable is all loaded and it has all the images in it from the batch one data batch one. And just to show we were talking about with the as type on there, if we go ahead and take X0 and just look for its max value, let me go ahead and run that. Uh you'll see it doesn't oops I said 128. It's 255. Uh you'll see it doesn't go over 255 because it's an basically an asky character is what we're keeping that down to. We're keeping those values down. So they're only 255 0 to 255 versus a float value which would bring this up um exponentially in size. And since we're using the map plot library, we can do um oops that's not what I wanted. Since we're using the map plot library, we can take our canvas and just do a plt. I im for image show. And let's just take a look at what x0 looks like. And it comes in I'm not sure what that is, but you can see it's a very lowgrade image uh broken down to the minimal pixels on there. And if we did the same thing, oh, let's do uh let's see what one looks like. Hopefully, it's a little easier to see. Run on there. Not enter. Let's hit the run on that. Uh and we can see this is probably a semitr. That's a good guess on there. And I can just go back up here instead of typing the same line in over and over. And we'll look at three. Uh that looks like a dump truck unloading. Uh and so on. You can do any of the 10,000 images. We can just jump to 55. Uh looks like some kind of animal looking at us there. Probably a dog. And just for fun, let's do just one more uh uh run on there. And we can see a nice car for our image number four. Uh so you can see we pace through all the different images. It's very easy to look at them. and they've been reshaped to fit our view and what the uh mapplot library uses for its format. So the next step is we're going to start creating some helper functions. We'll start by a one hot encoder to help us. We're processing the data. Remember that your labels, they can't just be words. They have to switch it and we use the one hot encoder to do that. And then we'll also create a uh class uh car helper. So it's going to have an init and a setup for the images. And then finally, we'll go ahead and run that code so you can see what that looks like. And then we get into the fun part where we're actually going to start creating our model, our actual neural network model. So let's start by creating our one hot encoder. We're going to create our own here. Uh and it's going to return an out and we'll have our vector coming in and our values equal 10. What this means is that we have the 10 values, the 10 possible labels. And remember, we don't look at the labels as a number because a car isn't one more than a horse. That'd be just kind of bizarre to have horse equals 0, car equals 1, plane equals 2, cat equals three. So a cat plus a car equals what? Uh so instead we create a numpy array of zeros and there's going to be 10 values. So we have a 10 different values in there. So you have uh zero or one. One means it's a cat. Zero means it's not a cat. Um in the next line it might be that uh one means it's a car, zero means it's not a car. So instead of having one output with a value of 0 to 10, you have 10 outputs with the values of 0 to one. That's what the one hot encoder is doing here. And we're going to utilize this in code in just a minute. So let's go ahead and take a look at the next helpers. We have a few of these helper functions we're going to build. And when you're working with a very complicated Python project, dividing it up into separate definitions and classes is very important. Otherwise, it just becomes really ungainainely to work with. So, let's go ahead and put in our next helper, uh, which is a class. And this there's a lot in this class. So, we we'll break it down here. Let's just start uh we can put a space right in there. There we go. That looks a little bit more readable. Add a second space. So, we're going to create our class, the cipher helper. And we'll start by initializing it. Now there's a lot going on in here. So let's start with the uh innit part. Uh self doi equals zero. That'll come in in a little bit. We'll come back to that in the lower part. We want to initialize our training batches. So when we went through this, there was like a meta batch. We don't need the meta batch, but we do need the data batch 1 2 3 4 5. And we do not want the testing batch in here. This is just the selfall train batches. So we're going to come make an array of of all those different images. And then of course we left the test batch out. So we have our self.est batch. Uh we're going to initialize the training images and the training labels and also the test images and the test labels. So these are just this is just to initialize these variables in here. Then we create another definition down here and this is going to set up the images. Let's just take a look and see what's going on in there. Now, we could have all just put this as part of the uh innit part uh since this is all just helper stuff. But breaking it up again makes it easier to read. It also makes it easier when we start executing the different pieces to see what's going on. So that way we have a nice print statement to say, hey, we're now running this and this is what's going on in here. We're going to set up these self-training images at this point and that's going to go to a numpy array vstack. And in there, we're going to load up uh in this case, the data for D and self, all trained batches. Again, that points right up to here. So, we're going to go through each one of these uh files or each one of these data sets because they're not a file anymore. We've brought them in. Data batch one points to the actual data. And so, our self-training images is going to stack them all into our into a numpy array. And then, it's always nice to get the training length. And that's just the total number of uh self-training images in there. And then we're going to take the self-training images. And let me switch marker colors because I am getting a little too much on the markers up here. Oops, there we go. Bring down our marker change so we can see it a little better. And at this point, this should look familiar. Where did we see this? Well, when we wanted to uh uh look at this above and we went to look at the images in the mattplot library, we had to reshape it. So, we're doing the same thing here. We're taking our self-training images and uh based on the training length, total number of images because we stacked them all together. So now it's just one large file of images. We're going to take and look at it as our our three video cameras that are each displaying a 32x32. We're going to switch that around so that now we have um each of our images that stays the same place. And then we have our 32x 32 and then by our three our last uh our three different values for the color. And of course we want to go ahead and uh they run this where we say divide by 255. That was from earlier. It just brings all the data into 0ero to one. That's what this is doing. So we're turning this into a 0 to one array which is uh all the pictures 32x 32x3. And then we're going to take the self-training labels and we're going to pump those through our one hot encoder we just made and we're going to stack them together. And uh again, we're converting this into an array that goes from uh instead of having horse equals 1, dog equals 2, and then horse plus dog would equal three, which would be cat. Now, it's going to be uh you know an array of 10 where each one is 0ero to one. Then we want to go ahead and set up our test images and labels. And uh when we're doing this, you're going to see it's the same thing we just did with the rest of them. Let me just change colors right here. This is no different than what we were doing up here with our training set. Uh we're going to stack the different uh images. Uh we're going to get the length of them so we know how many images are in there. Uh you certainly could add them by hand, but it's nice to let the computer do it, especially if it ever changes on the other end and you're using other data. And again, we reshape them and transpose them. And we also do the one hot encoder. same thing we just did on our training images. So now our test images are in the same format. So now we have a uh definition which sets up all our images in there. And then the next step is to go ahead and batch them or next batch. And let's do another breakout here for batches because this is really important to understand. Tends to throw me for a little loop when I'm working with TensorFlow or KAS or a lot of these. We have our data coming in. If you remember, we had like 10,000 photos. Let me just put 10,000 down here. We don't want to run all 10,000 at once. So, we want to break this up into batch sizes. And you also remember that we had the number of photos in this case, uh, length of test or whatever number is in there. Uh, we also have 32x 32x3. So when we're looking at the batch size, we want to change this from 10,000 to um a batch of in this case I think we're going to do batches of 100. So we want to look at just 100, the first hundred of the photos. And if you remember, we set selfi equal to zero. Uh so what we're looking at here is we're going to create x. We're going to get the next batch from the very initialized. We've already initialized it for zero. So we're going to look at x from zero to batch size, which we've set to 100. So just the first 100 images and then we're going to reshape that into uh and this is important to let the data know that we're looking at 100x 32x 32x3. Now we've already formatted it to the 32x 32x3. This just sets everything up correctly so that x has the data in there in the correct order in the correct shape. And then the y just like the x uh is our labels. So our training labels again they go from zero to batch size. In this case, they do self eye plus batch size because the self eye is going to keep changing. And then finally, we increment the self i because we have zero. So we so the next time we call it, we're going to get the next batch size. And so basically we have x and y. x being the photograph data coming in and y being the label. And that of course is labeled through one hot encoder. So if you remember correctly, if it was say horse is equal to zero, it would be one for the zero position since this is the horse. and then everything else would be zero in here. Let me just put lines through there. There we go. There's our array. Hard to see that array. So, let's go ahead and take that and uh we're going to finish loading it since this is our class. And now we're armed with all this um uh our setup over here. Let's go ahead and load that up. And so, we're going to create a variable ch with the car helper in it. And then we're going to do ch setup images. Uh now we could have just put all the setup images under the init but by breaking this up into two parts it makes it much more readable and um also if you're doing other work there's reasons to do that as far as the setup. Let's go ahead and run that. And you can see where it says uh setting up training images and labels setting up test images. And that's one of the reasons we broke it up is so that if you're testing this out you can actually have print statements in there telling you what's going on which is really nice. Uh they did a good job with this setup. I like the way that it was broken up in the back. And then one quick note, you want to remember that batch to set up the next batch is we have to run uh batch equals ch next batch of 100 because we're going to use the 100 size. Uh but we'll come back to that. We're going to use that. Just remember that that's part of our code we're going to be using in a minute from the definition we just made. So now we're ready to create our model. First thing we want to do is we want to import our TensorFlow as TF. I'll just go ahead and run that. So it's loaded up. And you can see we got a a warning here. Uh that's because they're making some changes. It's always growing and they're going to be depreciating one of the uh values from float 64 to float type or it's treated as an NP float 64. Uh nothing to really worry about cuz this doesn't even affect what we're working on because we've set all of our stuff to a 255 value or 0ero to one. And do keep in mind that 0ero to one value that we converted to 255 is still a float value. uh but it'll easily work with either the uh numpy float 64 or the numpy dtype float. It doesn't matter which one it goes through. So the depreciation would not affect our code as we have it. And in our tensorflow uh we'll go ahead. Let me just increase the size in there just a moment so you can get better view of the um what we're typing in. Uh we're going to set a couple placeholders here. And so we have we're going to set x equals tf placeholder tf float 32. We just talked about the float 64 versus the numpy float. We're actually just going to keep this at float 32 more than uh significant number of decimals for what we're working with. And since it's a placeholder, we're going to set the shape equal to and we've set it equal to none because at this point we're just holding the place on there. We'll be setting up as we run the batches. That's what that first value is. And then 32x 32x3. That's what we've reshaped our data to fit in. And then we have our y true equals placeholder tf float 32 and the shape equals none, 10. 10 is the 10 different labels we have. So it's an array of 10. And then let's create one more placeholder. We'll call this hold prob or hold probability. And we're going to use this. We don't have to have a shape or anything for this. This placeholder is for what we call dropout. If you remember from our theory before, we drop out so many nodes is looking at or the different values going through which helps decrease bias. So we need to go ahead and put a a placeholder for that also. And we'll run this. So it's all loaded up in there. So we have our three different placeholders. And since we're in TensorFlow, when you use KASS, it does some of this automatically, but we're in TensorFlow direct. Kas sits on TensorFlow. We're going to go ahead and create