AI in 2020

Key Takeaways

that Siraj guys never coming back hello world it's Suraj and AI in 2020 is once again going to dramatically shape the direction of human civilization in this episode I'll give 8 predictions about the field this year take a look at any of those top tech jobs of 2020 listicles from across the web almost every single one lists either machine learning engineer or data scientist as the number one most lucrative position these companies don't just want they need people trained in this skill set and the most exciting part is that unlike the other sciences many of them don't require an expensive degree that means that no matter what your background is or where you live in the world if you have the time and motivation you can seize this opportunity to not only solve problems at scale with machine learning but earn a sizable income and as an added bonus you'll probably make your parents proud mine are mostly my journey has been an emotional roller coaster for them but love you fam the best programming language to learn in 2020 is still Python Python is the lingua franca of machine learning and there's absolutely no sign that this will change there's some excitement around Swift because it just got ported to tensorflow and developers on Hacker News are excited about closure and rust but in the end Python is still the king the best machine learning library to learn is let's that's boba boba do you like tensorflow or pi torch PI torch judging by the researchers sentiment on social media Stack Overflow and github it just keeps increasing in popularity and one last thing before we get into the predictions the way to learn machine learning in 2020 is as follows first pick a problem that you'd like to solve be specific don't just choose cancer instead perhaps assessing risk factor biomarkers for stroke prevention then go to papers with code comm and search for relevant keywords most likely you will find a related paper and code the developers behind this app built an algorithm to crawl archive for the latest papers and automatically link them with their associated code now read the gist of the paper now go to the code repository on github download it and start working with it modify it experiment with it massage it when you encounter terms or concepts you don't understand like automatic differentiation or quantization just open a new tab and google the concept I promise you you will find some educational video or blog post about it basically learn by doing make sure to post your project on your github account when done and share it with other developers for feedback it's gonna look great to any prospective employers in the future and it's just fun alright let's get to the list my first prediction is that we'll see an increased emphasis by the research community on model robustness over accuracy this is not some god-given prediction I'm making by the way it's what almost every top researcher has been commenting on last year and at the annual AI research conference nerds thus far trying to outperform the state-of-the-art in accuracy by a few percentage points has been a top goal for many researchers and clickbait paper titles I mean seriously but as we learn more about this technology we're all realizing that accuracy alone is not a good metric to be aiming for the reality is much more nuanced than that explained ability power consumption data efficiency and reproducibility these are all equally valid goals to strive for neural networks are notoriously classified as black boxes outputting incredibly accurate predictions given some data but with no real way to understand why a great step in this direction is a paper from nur it's by Google titled neural tangents I also love that they added a hyperlink to the related collab notebook in their archive paper never seen that before neural tangents is a high-level Python library designed to enable more research into infinitely wide neural networks no one that works are no longer just matrix multiplications that nobody understands we are making progress towards a theoretical understanding of them consider a neural network where the number of neurons in each layer is increased to why because in mathematics it's usually easier to study concepts when stretched to an infinite limit without drugs when this happens we can consider the network as a function drawn from what are called Gaussian processes or GPS for any given data set there are potentially infinitely many functions we could use to fit it to the data GPS help solve this problem by assigning a probability value to each of these potential functions and in general this enables us to understand a huge range of phenomena in deep learning there's also xai an explained ability toolbox for machine learning by the Institute for ethical AI and ml it allows you to easily identify imbalances in data visualization correlations in terms of power consumption we're beginning to see how much AI data centers contribute to carbon emissions and that's not a sustainable long term strategy for our planet especially since the need for AI will only increase over time that's why quantization is a word that's on everybody's mind think of quantization like an umbrella term that describes a set of techniques used to convert input values from a big set to output values in a smaller set a new tool called graffities enables you to do just that it's a framework built on top of tensor flow to process low level graph descriptions of neural networks into efficient inference on fixed point Hardware there's also tensor flow lights and PI torch is always getting speed games from quantization one sound from my quantized networks big numbers imma quantize those trying to find those floating point numbers if for my networks Big O grows go train it just wants to be a train go training data efficiency was once again a super hot topic at Nurik since not everyone has access to huge data sets my favorite paper on this topic is called practical deep learning with Bayesian principles they showed how using Bayesian statistics has the potential to address issues like representing uncertainty using the data distribution and overfitting ultimately helping machines learn with less data and as for reproducibility Harbert and Google recently teamed up on writing deep network for seismic prediction no one could reproduce their results and it was later shown that the task could be done just as easily using a logistic regression model my second prediction is that most of the advances in AI this year aren't going to be in software they're going to be in hardware Nvidia is set to release their new 7 nanometer GPUs Google is gonna release its fourth-generation TP use Intel is gonna have their own GPUs but it's not just the big companies that are going to join in on the fun there are a ton of AI hardware startups that will finally release our products like cerebra systems as sumit centolla creator of pi torch recently said the next war is among compilers for the major frameworks XL ATVM pi torch has glow a lot of innovation is waiting to happen all right prediction three more multimodal and multitask learning multimodal learning is learning that involves using varied types of training data like images videos and text together instead of just one type we can see examples of this by looking at some of the biggest data sets released last year especially the autonomous driving ones from Wei MO and Baidu way Mo's open dataset contains not just millions of image frames from all of its driving but also related data like temperature pedestrian information geographical data and sergey brin social security number multitask learning is about having a model able to perform multiple tasks I personally think graph based networks are going to be a huge help here a paper from just last month titled an attention based graph neural network for heterogeneous structural learning points in this direction graph neural network theory is still in its early stages but the idea is to represent your network as a graph and have it solve graph related problems think social network analysis or the fastest way to get from point A to point B my fourth prediction is that we'll see way more neural symbolic architectures Gary Marcus who I once interviewed in Amsterdam is all about promoting symbolic AI instead of deep learning him and Yann laocoön debating this is basically a meme at this point the solution is to incorporate ideas from both factions symbolic AI was all the rage before deep learning came around basically the idea was to encode a representation of some object or concept using a series of human readable symbols neural networks also do this but in their own internal representational language the idea behind neuro symbolic AI is to combine the best of both worlds have neural networks to learn discrete symbols not just blackbox representations and use them for processing my fifth prediction is that machine learning operation skills are going to be valued more than model development skills this space has matured up to a point where training a model on some data can be done with a web browser a few lines of code or sometimes even with no code the more challenging aspects of machine learning then are going to be more valued that includes things like model access control database pipelines versioning performance monitoring continuous integration etc ml flow is a great library for this it helps you manage the whole machine learning lifecycle my sixth prediction is that the ethics of AI is going to be at the forefront of news headlines surrounding this technology a AI is used to manipulate people in the form of advertisements on almost every major platform it helps perform mass surveillance by governments and it automates the process of war who should get access to this power and how should we regulate it are two questions that will be top of mind for the community in 2020 just last year San Francisco became the first US city to ban facial recognition a bold move which will undoubtedly spark similar regulations in other cities this year my seventh prediction is that we are going to experience an absolute Renaissance and creativity because of advances in generative modeling technology considered the old if I the AI that could turn black and white pictures into colored pictures or how far we've come with AI generated voices or the fact that Stalag and two and D fakes are getting photorealistic totally indistinguishable from the real thing right now leverage these tools requires a little bit of programming knowledge but soon there are going to be one-click apps for all of these powers and in the hands of creators we're gonna see all new types of content emerge and my last prediction is that this is a year where the AI human collaborative deep blue moments happens and it's going to happen in drug discovery there are several reasons for that first the cost of computing keeps dropping aka Moore's law also the cost of genetic sequencing keeps dropping ai algorithms are improving the FDA recently approved a drug called Millison in ten months from idea to injection to help treat a young girl named Ulla suffering from a rare genetic disorder that is unprecedented usually drugs take at least a decade to get approved things are really accelerating in this space I personally can't stop thinking about me listen I see a huge opportunity to improve medicine with AI there's no question in my mind that AI applied to medicine is the most meaningful application of this technology so if you're wondering if AI is a good career trajectory to pursue this year the answer is definitively yes learn as much as you can believe in your ability to make a positive difference in the world and follow your heart I've got links to everything I've mentioned here for you in the video description happy learning wizards