Automatic Machine Learning

Key Takeaways

now I am become death the destroyer of worlds hello world it's Suraj an automatic machine learning or Auto ml is a field that's been gaining a lot of popularity within the AI community so I'm gonna explain how Auto ml works then compare the top auto ml frameworks to help you choose the best one for your needs whenever we have a data set and want to collect insights from it using machine learning there are a whole lot of steps that we have to complete like cleaning the data selecting the most relevant features choosing the right model not cringing from Ilan's awkward presentation of neural link in the right configurations or hyper parameters of that model this process can be long and expensive since it requires a lot of guessing and checking to ensure we have the best possible results the goal of Auto ml is to automate as many of these steps as possible without compromising the accuracy of our results the more steps we can automate the more there will be people who can actually use AI technology not just PhDs who have the necessary mathematical and programming skill sets the idea is to eventually drag and drop any data set into an app and have it perform all the steps necessary until it's a fully capable prediction model that can perform whatever task we've set for it to do but does that mean that there will someday no longer be a need for data scientists yeah so it's been real guys see ya know I kid absolutely not I like the analogy of being a chef in the kitchen chefs use various machines to help automate the more repetitive parts of their work like blending ingredients together so that they can focus on the more creative aspects of their work like deciding what to cook choosing the ingredients and what the larger process looks like in this way auto ml tools will help free up time for data scientists so they can focus on more creative things like how to properly frame a problem as a data science problem how to incorporate their domain knowledge how to interpret the results and how to communicate those results to the rest of their team I wrapped so far the main priority of the auto ml community has been developing techniques to automate two tasks in this pipeline model selection and hyper parameter selection that means finding the best performing statistical model for a given problem and the corresponding configurations without human intervention in terms of model selection there are lots of techniques out there but a technique that's particularly hot right now for neural network architectures is called differentiable architecture search or darts darts is a game I'm not very good at however darts the algorithm first assumes the space of candidate architectures the operations on the edges of the neural graph are unknown at first it then places mixtures of the disparate operations on each edge and jointly optimizes them by mixing the probabilities of each and the network weights are optimized by solving a gradient descent based problem which then gives you the final architecture from a mixture of those probabilities the results are impressive it's way more efficient than other neural architecture search algorithms in terms of hyper parameter selection this is a time intensive task that requires a lot of trial and error on the part of a data scientist since they basically have to guess some numbers then wait for their model to train itself on some data then reassess whether the results were ideal or not then redo the process with a better educated guess as to what the ideal numbers should be there are several techniques that are used to automate this process but no one technique is considered the best one two of the most widely used techniques are grid search and random search because they're so easy to implement look at that that's it I could code that in my sleep both techniques frame the problem mathematically consider a the space of possible strategies each of them simply picked several possible hyper parameters randomly try them out and then keep the most optimal performing ones in an iterative fashion these techniques are great for models with just a Hieu hyper parameters but when the search space gets bigger meeting more configurable numbers they can take way too long to compute so an alternative to these strategies for bigger models is to use meta heuristics these are a class of optimization techniques based on the famous exploitation versus exploration trade-off from machine learning theory in which an AI must decide whether to stay content by exploiting their existing resources or to find new resources by making a bet that what they will find is better than what they already have this applies to dating as well but I digress there's so many possible meta heuristic techniques out there simulated annealing is one of them it's inspired from the process of annealing in metallurgy that consists of alternating cooling and heating phases to minimize the material energy modifying its microstructure and physical properties by analogy we can explore the parameter space of our model to try to minimize a system energy our objective function another technique is to use particle swarms which sounds apocalyptic but it's not these use a population of candidate solutions or particles to explore the space a each particle moves in a depending on its current location it's best known past position and the whole populations best position we can tune the trade-off between exploitation and exploration by playing with the importance of these deferring factors let's also not forget evolutionary algorithms inspired by biological evolution we have a population of candidate solutions and we apply evolutionary laws to have them optimize the objective function called Fitness at each step or generation we select the fittest individuals the next generation is built via reproduction of these selected individuals reproduction is just some math operation that's defined and I have to name one more then we'll get to comparing frameworks Bayesian optimization my favorite shout out to my base wad it encodes prior values into the model usually based on domain knowledge in one example an objective function is approximated through a Gaussian process regression model the modeling techniques provide a probable the density function for the values of F based on priors at each step the new point to explore is selected as the maximum of a function called the activation function this function combines prior knowledge and the uncertainty about the function in order to optimize its high when there is a good chance that we'll find better solutions for our optimization problem basically Bayesian optimization starts by taking a point in the multi-dimensional space of hyper parameter configurations gets the corresponding objective function value and then selects a new point that minimizes the activation function this point is used to augment our data set and becomes a historical observation to be used in future point selections so these are some ways we can automate model selection and hyper parameter selection there exist several auto ml frameworks to help us do that and some of them even have the ability to automate other parts of the pipeline like data cleaning and feature engineering so let's start with the first one on my list ml box ml box is a Python framework that performs data cleaning hyper parameter selection and model selection it's already been tested on Kaggle and performs pretty well for example in the two sigma rental listing contest it helps someone rank top 5% I like how simple it is to install but it isn't as compatible with Mac and Windows as it is with Linux however the data cleaning functionality it offers is really impressive that's the most useful part of the library for example it can automatically merge several data sets into one so definitely use it for that aspect another is Otto cycle urn a framework built for the popular Python machine learning tool scikit-learn it performs model selection and hyper parameter tuning it also has feature engineering methods like one hot encoding and uses scikit-learn estimators for both classification and regression problems once it creates a pipeline it optimizes it using Bayesian search it works pretty well on small and medium sized data sets but can't be applied to modern deep learning systems that yield state-of-the-art performance on massive data sets sorry deep learning gods next up is teapot or the tree based pipeline optimization tool it optimizes pipelines using genetic programming which is pretty unique it again extends scikit-learn but with its own regressor and classify our methods it then explored thousands of possible pipelines before choosing the best one for your data the problem is that it can't automatically process natural language inputs and it's not able to process categorical strings which have to be integer coded before being passed in as data but I still like this one a lot look at all that github cloud also shout out to h2o it supports both traditional and deep learning model selection it uses its own algorithms to build a pipeline and includes feature engineering as well as hyper parameter tuning if you only want to do deep learning that's a great one auto chaos is a great pick because it follows the design of the classic scikit-learn api so it's simple to use but uses powerful neural architecture search with the Charis library under the hood to find the best model for the job Google recently released an entire suite of machine learning products called Auto ml in its cloud and uses transfer learning as well as neural architecture search on the backend to do this the plus is that it provides a pretty simple user interface to train and deploy models based on your own data the only problem is that it comes with a price tag which if you can pay great if not consider that there are open source alternatives out there that are free and last but not least ubers Ludwig a newer framework I actually made a video about a few months ago that aims to automate the entire deep learning pipeline with minimal or no code it's pretty much just a set of command-line arguments you'll be giving it but it's designed for deep learning not all machine learning models so keep that in mind you'll need lots of data to see good results before we talk about my favorites let's talk about the current limitations of Auto ml technology while it works really well for supervised learning algorithms that means when our data has labels we've yet to see it work well for a label less data also called unsupervised learning it also doesn't work well for reinforcement learning algorithms which are used in a time-based environment scenario like a real-world robot or a game the only real example I've seen of Auto ml in reinforcement learning is alpha zero Google's go playing AI that learned how to improve itself by playing against itself in that way it automatically learned its own model and hyper parameters Auto ml techniques also still can't handle complex data types lastly it's important to note that as of now feature engineering hasn't been automated that well that means the process of choosing what the ideal features to feed to a model for your problem still requires domain knowledge like knowing what the likely markers for a certain strain of cancer are or what coordinates in rural areas are likely to have certain mineral deposits so in conclusion if I were to pick my top three favorite auto ml frameworks for deep learning tasks its ubers Ludwig for traditional machine learning it's a tie between teapot and h2o thus there are three things to remember from this episode Auto ml refers to the process of automating some or all parts of the machine learning pipeline Ludwig teapot and h2o are my top three picks for auto ml frameworks and darts or differentiable architecture search is a model selection technique that definitely deserves more study link to it will be in the video description [Music]