AI Mastery Full Course | AI Tutorial for Beginners | Artificial Intelligence Course | Simplilearn

Key Takeaways

[Music] Welcome to our AI mastery full course by simply learn. Machines are now writing stories, creating art, and building apps just like humans do. What sounds like science fiction is happening right now. AI is changing everything from how doctors treat patients to how bank handle money and even how artists greet their work. Learning AI today isn't just a nice skill to have. It's your ticket to staying ahead in any career you choose. And that's why we made this course fun and super easy to understand. No boring textbook that puts you on sleep. Instead, you build cool real projects while learning step by step. We'll start super simple with an introduction to AI. Show exactly what path to take. Then we'll explore the exciting stuff like large language models, transformers, deep learning, and even multimodel AI that can understand text, images, and sound all at once. But here's what it really gets awesome. You'll actually create things people use. You'll build your own AI powered website that visitors will love, create iOS app with AI features, and make chat boards that can have real world conversation. And the coolest part is you'll even create an AI clone of yourself that can talk and act like you do. We don't just teach you the fun stuff. We prepare you for real world challenges, too. Where you learn how to handle AI bias, work on projects that solve actual problem, and master interview questions that can get you hired. Plus, we'll show you free AI tools for web scraping that give you superpowers in data collection. And by the time you finish, you won't just understand AI. You'll be the person who can build it and use it to solve real problems. So let's get started. Now before we move on, if you are interested in growing your career in AI, machine learning, this course is a great way to start. The professional certificate in AI and machine learning by simple learn university online and IBM will help you master all the key AI skills like charge GPT LMS deep learning agentic AI through live class hands-on learning in just 6 months where you'll be working on real world projects using 18 plus popular tools Python TensorFlow and earn certificates from Perdu and IBM you'll also get career support including resume help mock interviews and job assistance so whether If you are switching careers or upskilling, this course give you the latest AI knowledge and practical experience to stay ahead. So what are you waiting for? Hurry up and enroll now and you can find the course link below. >> Introduction of artificial intelligence and machine learning. By the end of this lesson, you will be able to define artificial intelligence, describe the relationship between artificial intelligence and data science, define machine learning, describe the relationship between machine learning, artificial intelligence, and data science. Describe different machine learning approaches. Identify the applications of machine learning. Let's understand how the field of artificial intelligence emerged. Let's first understand the reason behind the emergence of AI. Data economy is one of the factors behind the emergence of AI. It refers to how much data has grown over the past few years and how much more it can grow in the coming years. When you look at this graph, you can clearly understand how the volume of data has grown. You can see that since 2009, the data volume has increased by 44 times with the help of social websites. The explosion of data has given rise to a new economy and there is a constant battle for ownership of data between companies to derive benefits from it. Now that you know that data has grown at a rapid pace in the past few years and is going to continue to grow, let's understand the need for AI. As you know the increase in data volume has given rise to big data which helps manage huge amounts of data. Data science helps analyze that data. So the science associated with data is going toward a new paradigm where one can teach machines to learn from data and drive a variety of useful insights giving rise to artificial intelligence. Now you may ask what is artificial intelligence? Artificial intelligence refers to the intelligence displayed by machines that simulates human and animal intelligence. It involves intelligence agents, the autonomous entities that perceive their environment and take actions that maximize their chances of success at a given goal. Artificial intelligence is a technique that enables computers to mimic human intelligence using logic. It is a program that can sense, reason, and act. Let's look at some of the areas where artificial intelligence is used. Artificial intelligence is redefining industries by providing greater personalization to users and automating processes. One example of artificial intelligence in practice is self-driving cars. Self-driving cars are computercontrolled cars that drive themselves. In these cars, human drivers are never required to take control to safely operate the vehicle. These cars are also known as autonomous or driverless cars. Let's see how Apple uses AI. iPhone users can experience the power of Siri, the voice. It simplifies navigating through your iPhone as it listens to your voice commands to perform tasks. For instance, you can ask Siri to call your friend or to play music. Siri is fun and is extremely convenient to use. Another example is Google's Alph Go, which is a computer program that plays the board game Go. It is the first computer program to defeat a world champion at the ancient Chinese game of Go. Amazon Echo is another product. It's a home control chatbot device that responds to humans according to what they are saying. It responds by playing music, movies, and more. If you've got compatible smartome devices, you can tell Echo to dim the lights or turn appliances on or off. You can use AI and chess. And here is an example of a concierge robot from IBM called IBM Watson. The IBM Watson AI has typically been in the headlines for composing music, playing chess, and even cooking food. Let's move ahead and look at some sci-fi movies with the concept of artificial intelligence. The films featuring AI reflect the everchanging spectrum of our emotions regarding the machines we have created. Humans are fascinated by the concept of artificial intelligence and this is reflected in the wide range of movies on AI. Recommendations systems are used by a lot of e-commerce companies. Let's see how they work. Amazon collects data from users and recommends the best product according to the user's buying or shopping pattern. For example, when you search for a specific product in the Amazon store and add it to your cart, Amazon recommends some relevant products based on your past shopping and searching pattern. So before you buy the selected product, you get recommendations based on your interest and there is a possibility that you may also buy the relevant product with the selected product. If not, you have the chance to compare the selected product with the recommended products. Now let's move ahead and understand the relationship between artificial intelligence, machine learning, and data science. Even though the terms artificial intelligence, AI, machine learning, and data science fall in the same domain and are connected to each other, they have their specific applications and meaning. Let's try to understand a little about each of these terms. Artificial intelligence systems mimic or replicate human intelligence. Machine learning provides systems the ability to automatically learn and improve from the experiences without being explicitly programmed. Data science is an umbrella term that encompasses data analytics, data mining, machine learning, artificial intelligence, and several other related disciplines. Let's look at the flow diagram and try to understand the relationship between AI, machine learning, and data science. Interestingly, ML is also an element of artificial intelligence. So, the first step is data gathering and data transformation. This step basically comes under data science. Data transformation is the process of converting data from one format or structure into another format or structure. Data transformation is important to activities such as data management and data integration. After gathering data, we would want to use the data to make predictions and derive insights. In order to get predictions out of the data set, we use machine learning techniques such as supervised learning or unsupervised learning. On an overview level, supervised and unsupervised learning are the machine learning techniques used to extract predictions from a given data set. >> Now, you must be thinking where deep learning comes into the picture. Deep learning is a sub field of machine learning involved with algorithms. It uses artificial neural networks which are modeled on the structure and performance of neurons in the human brain. Deep learning is most effective when there isn't a clear structure to the data that you can just exploit and build features around. >> Now the next step in the flow diagram is to get insights from predictions being made. In order to do so, you need to use data analysis, which actually is the process under data science. Now, when you are done with all of these, you must want your data to perform some actions. This is where AI comes into the picture. Artificial intelligence combines predictions and insights to perform actions based on the human decision and automated decision. Now let's move ahead and understand the relationship between artificial intelligence, machine learning and data science. Let's look at the relationship between artificial intelligence and machine learning. Artificial intelligence is the engineering of making intelligent machines and programs. Machine learning provides systems the ability to learn from past experiences without being explicitly programmed. Machine learning allows machines to gain intelligence thereby enabling artificial intelligence. Let's now understand the relationship between machine learning and data science. Data science and machine learning go hand in hand. Data science helps evaluate data for machine learning algorithms. Data science covers the whole spectrum of data processing while machine learning has the algorithmic or statistical aspects. Data science is the use of statistical methods to find patterns in the data. Statistical machine learning uses the same techniques as data science. Data science includes various techniques like statistical modeling, visualization, and pattern recognition. Machine learning focuses on developing algorithms from the data provided by making predictions. So what is machine learning? Machine learning is the capability of an artificial intelligence system to learn by extracting patterns from data. It usually delivers quicker, more accurate results to help you spot profitable opportunities or dangerous risks. Now you must be curious to understand the features of machine learning. Machine learning uses the data to detect patterns in a data set and adjust program actions accordingly. Pattern detection can be defined as the classification of data based on knowledge already gained or on statistical information extracted from the patterns. It focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data by using a method called reinforcement learning. It uses external feedback to teach the system to change its internal workings in order to guess better next time. It enables computers to find hidden insights using iterative algorithms without being explicitly programmed. Machine learning uses algorithms that learn from previous data to help produce reliable and repeatable decisions. It automates analytical model building using the statistical and machine learning algorithms that tease patterns and relationships from data and express them as mathematical equations. Let's understand the different machine learning approaches. So what is the actual difference between traditional programming and machine learning? In traditional programming, data and program is provided to the computer. It processes them and gives the output. However, the machine learning approach is very different. In machine learning, algorithms are applied on the given data and output. The result of the applied algorithm and calculations is a learning model that helps machine to learn from the data. In traditional programming, you code the behavior of the program. But in machine learning, you leave a lot of that to the machine to learn from data. Now, let's first understand the traditional programming approach. Traditionally, you would hard-code the decision rules for a problem at hand, evaluate the results of the program, and if the results were satisfactory, the program would be deployed in production. If the results were not as expected, one would review the errors, change the program, and evaluate again. This iterative process continues till one gets the expected result. What is the machine learning approach? In the new machine learning approach, the decision rules are not hardcoded. The problem is solved by training a model with the training data in order to derive or learn an algorithm that best represents the relationship between the input and the output. This trained model is then evaluated against test data. If the results were satisfactory, the model would be deployed in production. And if the results are not satisfactory, the training is repeated with some changes. Machine learning techniques. Machine learning uses a number of theories and techniques from data science. Here are some machine learning techniques. Classification, categorization, clustering, trend analysis, anomaly detection, visualization, and decision making. Let's look at these techniques. Classification is a technique in which the computer program learns from the data input given to it and then uses this learning to classify new observations. Classification is used for predicting discrete responses. Classification is used when we are training a model to predict qualitative targets. Categorization is a technique of organizing data into categories for its most effective and efficient use. It makes free text searches faster and provides a better user experience. Clustering is a technique of grouping a set of objects in such a way that objects in the same group are most similar to each other than to those in other groups. It is basically a collection of objects on the basis of similarity and dissimilarity between them. Trend analysis is a technique aimed at projecting both current and future movement of events through the use of time series data analysis. It represents variations of low frequency in a time series. The high and medium frequency fluctuations being out. Anomaly detection is a technique to identify cases that are unusual within data that is seemingly homogeneous. Anomaly detection can be a key for solving intrusions by indicating a presence of intended or unintended induced attacks, defects, faults, and so on. Visualization is a technique to present data in a pictorial or graphical format. It enables decision makers to see analytics presented visually. When data is shown in the form of pictures, it becomes easy for users to understand it. Decisionmaking is a technique or skill that provides you with the ability to influence managerial decisions with data as evidence for those possibilities. Now I am sure you have a better understanding of the overview of machine learning. So let's look at some realtime applications of machine learning. Artificial intelligence and machine learning are being increasingly used in various functions such as image processing, robotics, data mining, video games, text analysis, and healthcare. Let's look at each of them in more details. So what is image processing? It is a technique to convert an image into a digital format and perform some operations on it so as to induce an enhanced image or to extract some helpful information from it. Let's look at some of the examples of image processing. Facebook does automatic face tagging by recognizing a face from a previous user's tagged photos. Another example is optional character recognition which scans printed docs to digitize the text. Self-driving cars are another big example of image processing. Autopilot is an optional drive system for Tesla cars. When autopilot is engaged, cars can self- steer, adjust speed, detect nearby obstacles, apply the brakes, and park. Now, let's see how robotics uses machine learning. Robots are machines that can be used to do certain jobs. Some of the examples of robotics are where a humanoid robot can read the emotions of human beings or an industrial robot is used for assembling and manufacturing products. So let's look at some realtime applications of machine learning. Let's see what data mining is. It is the method of analyzing hidden patterns in data. Let's look at some of the applications of data mining. It is used for anomaly detection to detect credit card fraud and to determine which transactions vary from usual purchasing patterns. It is also used in market basket analysis which is used to detect which items are often bought together. It can be used for grouping where it classifies users based on their profiles. Machine learning is also applied in many video games in order to give predictions based on data. In a Pokemon Go battle, there is a lot of data to take into account to correctly predict the winner of a battle. And this is where machine learning becomes useful. A machine learning classifier will predict the result of the match based on this data. Let's move on to one of the most popular applications of machine learning, which is text analysis. It is the automated process of obtaining information from text. One example of text analysis is spam filtering which is used to detect spam in emails. Another example is sentimental analysis which is used for classifying an opinion as positive, negative or neutral. It detects public sentiment in Twitter feed or filters customer complaints. It is also used for information extraction such as extracting specific data address, keyword or entities. There are many applications of machine learning in the healthcare industry identifying disease and diagnosis, drug discovery and manufacturing, medical imaging diagnosis and so on. Some of the companies that use machine learning have revolutionized the healthcare industry are Google Deep Mind Health, Biobeats, Health Fidelity, and Ginger.io. >> Today, we'll take you through the exciting road map of becoming an AI engineer. If our content psiques your interest and helps fuel your curiosity, don't forget to subscribe to our channel. Hit that bell icon so you never miss an update. Now let's embark on this AI journey together. As artificial intelligence continues to revolutionalize various industries, AI engineers stand at the forefront of this technological wave. These professionals are essential in crafting intelligence systems that address complex business challenges. AI projects often stumble due to poor planning, subpar architecture or scalability issues. AI engineers play a crucial role in overcoming these hurdles by merging cuttingedge AI technologies with strategic business insights. So in this video, we'll guide you through the essentials of becoming an AI engineer. Let's start with the basics. What does an AI engineer do? An AI engineer builds AI models using machine learning algorithms and deep learning neural networks. These models are pivotal in generating business insights that influence organizational decision making. From developing applications that leverage sentiment analysis for contextual advertising to creating systems for visual recognition and language translation, the scope of an AI engineer's work is vast and impactful. So to succeed as an AI engineer, you need a blend of technical progress and soft skills. So now let's break down this 8th month plan. Month one, computer science fundamentals and beginner's Python. So before we delve into AI, it's crucial to establish a strong foundation in computer science. This month you should focus on the following topics. Data representation. Understanding bits and bytes, how text and numbers are stored and a binary number system is foundational for everything in computing. This knowledge helps in comprehending how computers interpret and process data. Now next comes computer networks. Learn the basics of computer networks including IP addresses and internet routing protocols. It's essential to understand how data travels across networks using UDP, TCP and HTTP which form the backbone of the internet and the worldwide web. Next comes programming basics. Begin with the basics of programming like variables, strings, numbers, conditionals, loops, and algorithm basics. These fundamentals will allow you to write and understand simple programs. Simultaneously, you'll also start with Python, the preferred language for AI. So learn about variables, numbers, strings, lists, dictionaries, sets, pupils, and control structures like if conditionals and for loops. And then move on to functions and modules. Understand how to create functions including lambda functions and work with modules by using pip install to add functionality to your projects. Next comes file handling and exceptions. You should also practice reading from and writing to files as well as handling exceptions to make your programs more robust. finally grabs the basics of classes and objects which are crucial for writing organized and efficient code. So this comprehensive overview sets the stage for more complex programming tasks that you'll encounter in the following months. Now in month two, you'll move on to data structure algorithms and advanced Python. So building on the foundations from month one, we'll now delve into data structure and algorithm. So familiarize yourself with the concept of bigger notation to understand the efficiency of different algorithms and data structures. Learn about edits, link list, hash tables, stacks, cues, trees, and graphs. Mastering these structures will allow you to store and manipulate data effectively. Now, next comes algorithms. You should explore algorithms such as binary search, bubble sort, quick sort, merge sort, and recursion. These are essential for optimizing your code and parallelly you'll advance your Python skills. So, you can dive into inheritance, generators, iterations, list comprehensions, decorators, multi-threading, and multiprocessing. These topics will enable you to write more efficient and scalable code. So this month's learning prepares you to handle complex data operations and enhance your coding efficiency. Now in month three, you'll move on to version control, SQL and data manipulation. So in the third month, the focus shifts to collaboration and data management. Number one, version control. So understand the importance of version control systems, especially Git and GitHub. So learn basic commands such as add, commit, and push. You should also learn how to handle branches, reward changes and understand concepts like head diff and merge. So these skills are invaluable for tracking changes and collaborating with other developers. Next, pull requests. Master the art of creating and managing pull requests to contribute to collaborative projects. Next, we'll dive into SQL for managing databases. So first we'll start with SQL basics. So learn about relational databases and how to perform basic queries and then you'll move on to advanced queries. Understand complex query techniques such as CT, subqueries and window functions. And then comes joins and database management. So study different types of joins like left, right, inner and full joint. You should also learn how to create databases, manage indexes and write store procedures. Additionally, you'll use numpy and pandas for data manipulation and learn basic data visualization techniques. This comprehensive skill set will be crucial as you move into more advanced data science topics. So now in month four you'll deal with maths and statistics for AI. So mathematics and statistics are the backbone of AI and this month is dedicated to these critical subjects. So first learn about descriptive versus inferial statistics, continuous versus discrete data, nominal versus ordinal data, measures of central tendency like mean, median, mode and measures of dispersion like variance and standard deviation. After that understand the basis of probability and delve into normal distribution, correlation and coariance. After which you should move on to advanced concepts. So you can study the central limit theorem, hypothesis testing, p values, confidence intervals and so on. In path you should also study linear algebra and calculus. So in linear algebra learn about vectors, matrices, ean values and ean vectors. And in calculus, cover the basics of integral and differential calculus. So this mathematical foundation is essential for developing and understanding AI models setting you up for success as you transition into machine learning. Now in month five comes exploratory data analysis which is EDA and machine learning. So with a solid foundation in math and statistics, you are now ready to delve into machine learning. Number one, pre-processing. Learn how to handle NA values, treat out layers, perform data normalization, and conduct feature engineering. You should also understand encoding techniques such as one hot and label encoding. You'll also explore supervised and unsupervised learning with a focus on regression and classification and learn about linear models like linear and logistical regression and nonlinear models like decision tree random forest etc. And then understand how to evaluate models using metrics such as mean squared error mean absolute error mean for regression and accuracy precision recall etc. Then comes hyperparameter tuning. Learn about techniques like grid search CV and random search CV for optimizing your models. After which you'll move on to unsupervised learning. Here you can study clustering techniques like K means and hierarchical clustering and delve into dimensionality reduction with PCA. So this month's focus on EDA and model building will prepare you for more complex AI applications. Transitioning to the next phase, you'll begin to work on deploying these models and real world scenarios. So in month six comes MLOps and machine learning projects. So this month we'll cover the operational aspects of machine learning and work on practical projects. So in MLOps basics learn about APIs particularly using fast API for Python and server development. Understand DevOps fundamentals including CI/CD pipelines and containerization with Docker and Kubernetes. You should also gain familiarity with at least one cloud platform like AWS or Azure. Now in month seven comes deep learning. So in this month we delve into the world of deep learning. So number one comes neural network. So learn about neural networks including forward and backward propagation and build multi-layer perceptrons after which you'll move on to advanced architectures. So here explore convolutional neural networks which are CNN's for image data and sequence models like RNNs and LSTMs. So this deep learning knowledge will be crucial as you move into specialized areas of AI in the final month. Now in the final month, the eighth month comes NLP or computer vision. So the final month you have the option to specialize in either natural language processing, NLP or computer vision. So first we'll NLP track. So here you should learn about rejects text representation methods like count vectorizer TF b word to vec embeddings and text classification with nave base and familiarize yourself with the fundamentals of libraries like spacy and NLTK and work on end toend NLP project. And talking about computer vision track focus on basic image processing techniques like filtering, edge detection, image scaling and rotation. Utilize libraries like OpenCV and build upon the CNN knowledge from the previous month. Practice data prep-processing and augmentation. So by the end of this month, you should have a solid foundation in your chosen specialization ready to embark on your AI engineering career. So in conclusion, adopting AI is more than just a trend. It's a strategic move that can transform your organization's approach to machine learning. So do check out our professional certificate course in AI and machine learning which is designed to equip you with the skills needed to excel in this dynamic field. Through live classes led by industry experts, master classes from IIT Kur faculty and hands-on projects, you'll gain a competitive edge in AI. To enroll in this program, candidates should have a bachelor's degree with a 50% or higher average, preferably two or more years of work experience and foundational knowledge in programming and mathematics. Check out the course link in the description box below. So that's all for today's video on becoming an AI engineer. We hope you found this guide valuable and informative. >> Why reinforcement learning? Training a machine learning model requires a lot of data which might not always be available to us. Further, the data provided might not be reliable. Learning from a small subset of actions will not help expand the vast realm of solutions that may work for a particular problem. And you can see here we have the robot learning to walk. um very complicated setup when you're learning how to walk and you'll start asking questions like if I'm taking one step forward and left, what happens if I pick up a 50 pound object, how does that change how a robot would walk? These things are very difficult to program because there's no actual information on it until it's actually tried out. Learning from a small subset of actions will not help expand the vast realm of solutions that may work for a particular problem. And we'll see here it learned how to walk. This is going to slow the growth that technology is capable of. Machines need to learn to perform actions by themselves and not just learn off humans. And you see the objective climb a mountain. Real interesting point here is that as human beings, we can go into a very unknown environment and we can adjust for it and kind of explore and play with it. Most of the models, the non-reinforcement models in computer u machine learning aren't able to do that very well. Uh there's a couple of them that can be used or integrated. See how it goes is what we're talking about with reinforcement learning. So what is reinforcement learning? Reinforcement learning is a subbranch of machine learning that trains a model to return an optimum solution for a problem by taking a sequence of decisions by itself. Consider a robot learning to go from one place to another. The robot is given a scenario must arrive at a solution by itself. The robot can take different paths to reach the destination. It will know the best path by the time taken on each path. It might even come up with a unique solution all by itself. And that's really important is we're looking for unique solutions. Uh we want the best solution, but you can't find it unless you try it. So we're looking at uh our different systems, our different model. We have supervised versus unsupervised versus reinforcement learning. And with the supervised learning, that is probably the most controlled environment. Uh we have a lot of different supervised learning models where there's linear regression, neural networks, um there's all kinds of things in between, decision trees. The data provided is labeled data with output values specified. And this is important because we talk about supervised learning. You already know the answer for all this information. You already know the picture has a motorcycle in it. So you're supervised learning. You already know that um the outcome for tomorrow for you know going back a week. You're looking at stock. You can already have like the graph of what the next day looks like. So you have an answer for it. And you have labeled data which is used. You have an external supervision and solves problems by mapping labeled input to known output. So very controlled unsupervised learning and unsupervised learning is really interesting because it's now taking part in many other models. They start with an you can actually insert an unsupervised learning model um in almost either supervised or reinforcement learning as part of the system which is really cool. Uh data provided is unlabeled data. The outputs are not specified. machine makes its own predictions used to solve association with clustering problems. Unlabeled data is used. No supervision. Solves problems by understanding patterns and discovering output. Uh so you can look at this and you can think um some of these things go with each other. They belong together. So it's looking for what connects in different ways. And there's a lot of different algorithms that look at this. Um when you start getting into those there's some really cool images that come up of what unsupervised learning is. How it can pick out say uh the area of a donut. One model will see the area of the donut and the other one will divide it into three sections based on its location versus what's next to it. So there's a lot of stuff that goes in with unsupervised learning. And then we're looking at reinforcement learning. Probably the biggest industry in today's market uh in machine learning or growing market. It's very in its very infant stage uh as far as how it works and what it's going to be capable of. The machine learns from its environment using rewards and errors used to solve rewardbased problems. No predefined data is used. No supervision follows trail and error problem solving approach. Uh so again we have a random at first you start with a random I try this it works and this is my reward. doesn't work very well maybe or maybe doesn't even get you where you're trying to get it to do and you get your reward back and then it looks at that and says well let's try something else and it starts to play with these different things finding the best route. So let's take a look at important terms in today's reinforcement model and this has become pretty standardized over the last uh few years. So these are really good to know. We have the agent uh agent is the model that is being trained via reinforcement learning. So this is your actual u entity that has however you're doing it whether you're using a neural network or a Q table or whatever combination thereof. This is the actual agent that you're using. This is the model and you have your environment. Uh the training situation that the model must optimize to is called its environment. Uh and you can see here I guess we have a robot who's trying to get a chest full of gyms or whatever. And that's the output. And then you have your action. This is all possible steps that can be taken by the model. And it picks one action. And you can see here it's picked three different uh routes to get to the chest of diamonds and gems. We have a state. the current position condition returned by the model. And you could look at this uh if you're playing like a video game, this is the screen you're looking at. Uh so when you go back here, uh the environment is the whole game board. So if you're playing one of those Mobius games, you might have the whole game board going on. Uh but then you have your current position. Where are you on that game board? What's around that? What's around you? Um, if you were talking about a robot, the environment might be moving around the yard, where it is in the yard and what it can see, what input it has in that location. That would be the current position condition returned by the model. And then the reward uh to help the model move in the right direction, it is rewarded. Points are given to it to appraise some kind of action. So, yeah, you did good or if uh didn't do as good, trying to maximize the reward and have the best reward possible. And then policy. Policy determines how an agent will behave at any time. It acts as a mapping between action and present state. This is part of the model. What what is your action that you're you're going to take? What's the policy you're using to have an output from your agent? One of the reasons they separate a policy as its own entity is that you usually have a prediction um of a different options and then the policy well how am I going to pick the best based on those predictions I'm going to guess at different options and we'll actually weigh those options in and find the best option we think will work. Uh so it's a little tricky but the policy thing is actually pretty cool how it works. Let's go ahead and take a look at a reinforcement learning example. And just in looking at this, we're going to take a look uh consider what a dog um that we want to train. Uh so the dog would be like the agent. So you have your your puppy or whatever. Uh and then your environment is going to be the whole house or whatever it is where you're training them. And then you have an action. We want to teach the dog to fetch. So action equals fetching. Uh and then we have a little biscuit. So we can get the dog to perform various actions by offering incentives such as a dog biscuit as a reward. The dog will follow a policy to maximize this reward and hence will follow every command and might even learn new actions like begging by itself. Uh so you have b you know so we start off with fetching it goes oh I get a biscuit for that. it tries something else and you get a handshake or begging or something like that and it goes oh this is also rewardbased and so it kind of explores things to find out what will bring it as biscuit and that's very much like how a reinforced model goes is it uh looks for different rewards how do I find can I try different things and find a reward that works the dog also will want to run around and play and explore it environment uh this quality of model is called exploration so there's a little randomness going on in exploration and explores new parts of the house. Climbing on the sofa doesn't get a reward. In fact, it usually gets kicked off the sofa. So, let's talk a little bit about Marov's decision process. Uh Marov's decision process is a reinforcement learning policy used to map a current state to an action where the agent continuously interacts with the environment to produce new solutions and receive rewards. And you'll see here's all of our different uh uh vocabulary we just went over. We have our reward, our state, our agent, our environment interaction. And so even though the environment kind of contains everything um that you you really when you're actually writing the program, your environment is going to put out a reward and state that goes into the agent. Uh the agent then looks at this uh state or it looks at the reward usually um first and it says okay I got rewarded for whatever I just did or I didn't get rewarded and then it looks at the state and then it comes back and if you remember from policy the policy comes in um and then we have a reward. The policy is that part that's connected at the bottom. And so it looks at that policy and it says, "Hey, what's a good action that will probably be similar to what I did?" Or um uh sometimes they're completely random, but what's a good action that's going to bring me a different reward? So, taking the time to just understand these different pieces as they go is pretty important in most of the models today. Um, and so a lot of them actually have templates based on this that you can pull in and start using. Um, pretty straightforward as far as once you start seeing how it works. Uh, you can see your environment send it says, "Hey, this is the agent did this. If you're a character in a game, this happened and it shoots out a reward in a state." The agent looks at the reward, looks at the new state, and then takes a little guess and says, "I'm going to try this action." And then that action goes back into the environment. it affects the environment. The environment then changes depending on what the action was and then it has a new state and a new reward that goes back to the agent. So in the diagram shown, we need to find the shortest path between node A and D. Each path has a reward associated with it and the path with a maximum reward is what we want to choose. The nodes A, B, C, Denote the nodes to travel from node uh A to B is an action. Reward is the cost of each path and policy is each path taken. And you can see here A can go uh to B or A can go to C right off the bat or it can go right to D. And if you explored all three of these uh you would find that A going to D was a zero reward. Um A going to C and D would generate a different reward. Or you could go AC B D. There's a lot of options here. Um and so when we start looking at this diagram, you start to realize that even though uh today's reinforced learning models do really good at um finding an answer, they end up trying almost all the different directions you see. And so they take up a lot of work uh or a lot of processing time for reinforcement learning. They're right now in their infant stage and they're really good at solving simple problems and we'll take a look at one of those in just a minute in a tic-tac-toe game. Uh but you can see here uh once it's gone through these and it's explored, it's going to find the AC D is the best reward. It gets a full 30 points for it. So let's go ahead and take a look at a reinforcement learning demo. Uh and in this demo, we're going to use reinforcement learning to make a tic-tac-toe game. You'll be playing this game against the machine learning model, and we'll go ahead and we're doing it in Python. So, let's go ahead and go through um I always uh not always actually have a lot of Python tools. Let's go through um Anaconda, which will open up a Jupyter notebook. Seems like a lot of steps, but it's worth it to keep all my stuff separate. And it's also has a nice display when you're in the Jupyter notebook for doing Python. So, here's our Anaconda Navigator. I open up the notebook, which is going to take me to a web page. And I've gone in here and created a new uh Python folder. In this case, I've already done it and enabled it. Change the name to tic-tac-toe. Uh, and then for this example, uh, we're going to go ahead and import a couple things. We're going to, um, import numpy as np. We'll go ahead and import pickle. Numpy, of course, is our number array. And then, uh, pickle is just a nice way sometimes for storing, uh, different information, uh, different states that we're going to go through on here. Uh, and so we're going to create a class called state. I'm going to start with that. And there's a lot of lines of code to this uh class that we're going to put in here. Don't let that scare you too much. There's not as much here. Um it looks like there's going to be a lot here, but there really is just a lot of setup going on in the in our class state. And so we have up here, we're going to initialize it. Um we have our board. Um, it's a tic-tac-toe board, so we're only dealing with nine spots on the board. Uh, we have player one, player two, uh, is end. We're going to create a board hash. Um, we'll look at that in just a minute. We're just going to store some information in there. Symbol of player equals 1. Um, so there's a few things going on as far as the initialization. Uh, then something simple. We're just going to get the hash um of the board. You get the information from the board on there, which is uh columns and rows. We want to know when a winner occurs. Uh so if you get three in a row, that's what this whole section here is for. Uh let me go ahead and scroll up a little bit. And you can get a copy of this code if you send a note over to SimplyLearn. We'll send you over um this particular file and you can play with it yourself and see how it's put together. I don't want to spend a huge amount of time on this uh because this is just some real general Python coding. Uh but you can see here we're just going through um all the rows and you add them together and if it equals three, three in a row. Same thing with columns. U diagonal. So you got to check the diagonal. That's what all this stuff does here is it just goes through the different areas. Actually, let me go ahead and put There we go. Um, and then it comes down here and we do our sum and it says true. Uh, minus three. It just says did somebody win or is it a tie? So, you got to add up all the numbers on there anyway just in case they're all filled up. And next, we also need to know available positions. Um, these are ones that don't no one's ever used before. This way, when you try something or the computer tries something, uh, it's not going to give it an illegal move. That's what the available positions is doing. Uh then we want to update our state. And so you have your position going in. We're just sending in the position that you just chose. And you'll see there's a little user interface we put in there. We p pick the row and column in there. And again, I mean, this is a lot of code. Uh so really, it's kind of a thing you'd want to go through and play with a little bit and just read through it, get a copy of it. Uh great way to understand how this works. And here is a given reward. Um, so we're going to give a reward. Result equals self-winner. This is one of the hearts of what's going on here. Uh, is we have a result self.winner. So if there's a winner, then we have a result. If the result equals one, here's our feedback. Uh, if it doesn't equal one, then it gets a zero. So it only gets a reward in this particular case if it wins. And that's important to know because different uh systems of reinforced learning do rewarding a lot differently depending on what you're trying to do. This is a very simple example with a 3x3 board. Imagine if you're playing a video game. Uh certainly you only have so many actions, but your environment is huge. You have a lot going on in the environment. And suddenly a reward system like this is going to be just um is going to have to change a little bit. is going to have to have different rewards and different setup. And there's all kinds of advanced ways to do that as far as weighing you add weights to it. And so they can add the weights up depending on where the reward comes in. So it might be that you actually get a reward. In this case, you get the reward at the end of the game. And I'm spending just a little bit of time on this because this is an important thing to note. But there's different ways to add up those rewards. it might have like if you take a certain path um the first reward is going to be weighed a little bit less than the last reward because the last reward is actually winning the game or scoring or whatever it is. So this reward system gets really complicated on some of the more advanced uh setups. Um in this case though you can see right here that they give a a 0.1 and a 0.5 reward um just for getting a picking the right value and something that's actually valid instead of picking an invalid value. So rewards again that's like key that's huge. How do you feed the rewards back in? Uh then we have a board reset. That's pretty straightforward. It just goes back and resets the board to the beginning because it's going to try out all these different things while it's learning. It's going to do it by trial and error. So, you have to keep resetting it. And then, of course, there's the play. We want to go ahead and play uh rounds equals 100. Depends on what you want to do on here. Um you can set this different. You obviously set that to higher level, but this is just going to go through and you'll see in here uh that we have player one and player two. This is this is the computer playing itself. Uh, one of the more powerful ways to learn to play a game or even learn something that isn't a game is to have two of these models that are basically trying to beat each other. And so they always they keep finding explore new things. This one works for this one. So this one tries new things. It beats this. We've seen this in um chess I think was a big one where they had the two players in chess with reinforcement learning. uh is one of the ways they train one of the top um computer chess playing algorithms. Uh so this is just what this is. It's going to choose an action. It's going to try something and the more it tries stuff um the more we're going to record the hash. We actually have a board hash where they self get the hash set up on here where it stores all the information. And then once you get to a win, one of them wins, it gets the reward. Uh then we go back and reset and try again. And then kind of the fun part we actually get down here is uh we're going to play with a human. So we'll get a chance to come in here and see what that looks like when you put your own information in. And then it just comes in here and does the same thing it did above. It gives it a reward for its things um or sees if it wins or ties. Um looks at available positions, all that kind of fun stuff. And then finally, we want to show the board. Uh so it's going to print the board out each time. Really um as an integration is not that exciting. What's exciting uh in here is one looking at this reward system. Whoops. Play one more up. The reward system is really the heart of this. How do you reward the different uh setup and the other one is when it's playing it's got to take an action. And so what it chooses for an action is also the heart of reinforcement learning. How do we choose that action? And those are really key