AI Weekly Update #12 - November 18th, 2019

Key Takeaways

thanks for watching a weekly update from Henry AI labs this week we'll cover a lot of interesting things like a new image net classification state of the art and a lot of interesting articles and tutorials around model distillation and self supervised learning we'll also look at it really interesting write-up from Jeff Dean on the deep learning Revolution and the advances and hardware and the future of chip design thanks for watching and please subscribe to Henry a allows for more deep learning in AI videos this AI weekly update we'll begin by looking at the new imagenet classification state of the art presented by Google AI with a novel approach to model distillation then we'll look further at model distillation with this great article and code tutorial from Floyd help distilling knowledge from neural networks to build smaller and faster models particularly in the context of natural language processing for model distillation then we'll look at this great write-up from Jeff Dean the deep learning Revolution and its implications for computer architecture and chip design discussing some of the history of the slowing of Moore's law AI and compute and discussing a lot of interesting directions for the future of chip design and deep learning and machine learning then from google's AI blog we'll look at discussion about mobile net b3 and the mobile metallized for deployment on the edge TPU and discussion about Hardware where neural architecture search for the sake of faster inference on these mobile devices the sweetie update will also cover this great blog post from lillian wang on self supervised representation learning going through a lot of the different self supervised learning tasks that have been designed for images videos and reinforcement learning then we'll look at this great article on accelerating ren forests by up to 45 times using Co ml a discussion on how the random forest algorithm is embarrassingly parallel and how it can easily be distributed across different cores the GPUs and accelerated by using the rapid comm library then from Facebook will look at new progress and using AI to detect harmful content things like the self supervise xlm model in order to make use of English to improve detection in foreign languages as well then from Stanford will look at controlling assistive robots with learn enlightened actions this is the idea of having these low dimensional codes that can be used to control the high dimensional degrees of freedom of the assistive robotic arms then we look at the use of convolutional and recurrent neural networks to do epilepsy prediction from this EEG signal data then when Google will look at this new approach to self supervised learning for pitch estimation and then its application to separating different audio signals from audio files then we'll look at this paper specializing word embeddings for parsing by information bottleneck which one the best paper warned the e/m and NLP conference in from nvidia we'll look at this company Harrison AI and their success in predicting in vitro fertilization and different applications and medical imaging also from in video we'll look at Facebook's fashion plus plus model and a new video further explaining the algorithm then we'll also look at a discussion on mixed precision computing from Nvidia and we'll look at this new Calvin library that is facilitating 3d deep learning research at PI Kirk's library that makes it easier to upload these different kinds of things like voxels point clouds or these meshes to do 3d deep learning and 3d computer vision then from the batch will look at all these different articles that they presented and a different and a note on doing machine learning with small data one of the most interesting updates to AI this week is a new state of the art image net classification at 87.4% top one accuracy to noting a 1% improvement over the previous AVR this technique uses something called knowledge or model distillation the idea here is that you have a teacher network that is usually a really high large capacity model such as an efficient that be seven or any of these kinds of massive models and use this in order to smoothen out the labels on the data set and make it easier to distill the knowledge or train a student network usually this is done with a smaller capacity student network and it's frequently associated with speeding up inference reducing memory cost and these kinds of things so the idea is basically if you say you have the CFR ten data set when you have one denoting say cat and then zero everywhere else you would run the efficient net be seven or some kind of higher capacity model to predict the label distribution on that image and rather than having one for cat and zero everywhere else you'd get this distribution which is called a soft distribution where we have sadie 0.95 for cat and then 0.04 for dog and then say for something like ship or truck you might have like zero point zero zero zero zero one and so doing this kind of label smoothing essentially helps the student network get a better sense of the data distribution and it's better than any kind of prior you can enforce on the label smoothing so in this paper self training with noisy student improves image net classification instead of having the teacher-student network be higher capacity than the student network they actually are gonna have larger and larger models where the teacher has lower capacity than the student up until the end where it's they have the same capacity so another interesting thing about this is that they're using a different pipeline so rather than have the teacher now workers run its predictions on the original dataset they're also going to use this large unlabeled 300 million jft dataset so they're gonna have the original image net data set the teacher network is trained on and then it's gonna run its predictions to pseudo label the unlabeled data set so this is really similar to the semi-weekly supervised training framework from facebook where they're doing a similar kind of pipeline but they're labeling their relabeling the 3.5 billion images that were previously weakly labeled by using the hashtags as people upload the photos to Instagram so a lot of other interesting things in this paper is the use of noise things like having dropouts stochastic depth and data augmentation to improve the model and then exactly the pipeline the algorithm for how they do this also interesting Lee is this kind of robustness for free that comes up with it shown in this table and then I think another really interesting detail for this paper is the scaling up of the efficient net family so basically they go from a series of efficient net models so they go from B 1 B 2 B 3 when they're doing the teacher-to-student and I think that this is a really significant part of this pipeline and you see a lot of these other papers that say things like in the Distilled Burt where there's some kind of correlation between the family architectures so it's definitely really interesting paper and definitely something worth checking out your further ly interested in my take on this paper please check out the video I posted on the Henry air labs YouTube channel self training with noisy student 87.4% imagenet top one accuracy getting a little more into the details of this paper slow F has published a really great tutorial describing model distillation and then having a code example for doing the distilling bird into a by LCM model in this article distilling knowledge from neural networks to build smaller and faster models so the article begins with the description of distillation and the description of how you have these original labels that have been been smoothed by using the teacher network so for example in the task of predicting the next word you might have a sentence like the quick brown fox jumps over and then you would mask out this word and have the language model predict the master at work so you can see already how having one hot labels over these masked out words doesn't make a lot of sense compared to having these soft label distributions if the teacher producers for the student network so a good example of this in the jumps example is that you can have all these different words that precede jump jump over jump to jump though or Jim forty two for Jim for these new times just showing that natural language is very unpredictable and it's really hard to have these kind of one hot labels for learning this kind of next word task so in the article that then detail this pipeline you have the preacher and teacher model then you fine-tune it with the training set and then you use this to have the new label distribution that you use to train the student although the student has this kind of Alpha and the one minus alpha I think it's supposed to be ordered parameters to wait the hard labels or the you know the one zero zero zero kind of label compared with the soft labels produced by the teacher so also in the article they discussed this kind of idea of having a family of architectures in the hugging phase implementation of this in dis stillbirth they use a smaller transformer model so it's not the same as a bi LCM a transformer model is basically the idea of Burt it's just a smaller version of kind of the Burt architecture so in this in these paragraphs the author describes how this kind of family of architectures is important for model distillation then the article discusses using data augmentation in this pipeline and how you can use the pre-trained language model to construct new sentences in ways that make sense compared to say just randomly swapping out now it's pronounced because you might corrupt the label of the sentence the article then goes on to discuss the idea of using temperature in this framework so the idea of temperature is that usually you have this kind of softmax distribution they use in order to have the labels kind of sum up to one and you could add this temperature which is included in the original paper by geoff hinton oreal venules and Jeff Dean about how you add this temperature term to the top and the bottom of the softmax function in order to kind of smooth it out this distribution and make sure it's not too biased and is you know informative enough so then the article goes into the code I've shown you how to use these things like the fast text word vectors and all these different kinds of things if you want or interested in doing this yourself and then in the kind of conclusion of the study the author discusses having a sixty five times less computation to run inference on a single sentence with the final model and then 57 times later in the terms of the trainable parameters and then also in an ablation study the author finds that the use of data augmentation is enormous for the success of this model as well as many other things related to the experiment things like how many augment examples are produced and different lessons learned that the author you know through doing this example so it's definitely a really interesting article one of the best descriptions of model knowledge distillation in the context of NLP and definitely a great thing if you're interested in getting started with this for yourself the deep learning revolution and its implications for computer architecture and chip design is a really great write-up from Jeff Dean following his keynote at the 2020 international solid-state circuits conference this article discusses a lot of interesting things about machine learning and then the computational devices that run machine learning so it starts off with a discussion about the advances in machine learning things like the advanced and imagenet classification accuracy you see the trend towards solving imagenet which is really interesting and then the article goes on to talk about these interesting trends like the slowing of Moore's Law the computation done by open AI and their article computing a I think is the title of it where they showed the doubling time and computational demand every three point four months of some major contributions in AI and then also interestingly the growing rate of more people publishing papers in machine learning and these kinds of areas so this graph shows the slowing of the Moore's law this graph shows from the open ai ai and compute the increase in compute used for a lot of the recent models like neuro machine translation neural architecture search and alphago zero and then some graph about the new machine learning papers that are being published so the article then goes on to have this interesting discussion about trying to deploy these kind of speech recognition models required to scaling up the data center like crazy so they look into specialized hardware inspiring the design of first the tbu version one which is the single chip designed to target inference acceleration I've discussed the use of lower precision weights using 8-bit integers in the TPU version 1 and then some of different you know performance gains 15 to 30 times faster than the GPU or CPU at the time and then you know having more efficiency with you know computations for power usage so then the article goes on to discuss different chip companies like Cerberus graph core and Nirvana different you know exciting things that are being developed for this kind of hardware specialized deep learning like inference and training assessing the differences the challenge between designing training chips compared to inference chips with designing training chips being a little more challenging and then it goes on into a more specific article on both precision numeric formats and kind of the idea of reducing for infants acceleration and then also interestingly is this discussion of how the needs to be kind of like since chip design projects take about 18 to 24 months to finish the design and then bring them into a data center environment it's you need to have this kind of code design where you have people who are kind of forecasting the future trends and machine learning to kind of see what kind of design of the chips will be best and particularly in this article discussing the future potential these kinds of mixture of experts models and the need to have this kind of co.design with the mixture of experts networks although it's definitely interesting to think about all the different ways that this could play out and the different kinds of futures for the chip designs to accommodate the future of these neural network architectures so then the article goes on to using machine learning for chip design you know having cases like where you use reinforcement learning to play the game and placement and routing similar to like how deep mind did the data center control or you use this kind of machine learning algorithm in order to solve these np-hard optimization problems so also in the article it goes on to discuss like data structures and how you can do these learned index structures and how this compares things like you know b-trees hash tables and these different kinds of data structures so it's definitely a really interesting article also discussing the mixture of experts model which is this model that is kind of sparsely activated so you have this big neural network and then when a task comes in the model knows how to route it through the big neural network to produce different outputs and more efficient higher performance but it definitely a really interesting model that I think is has a lot of potential obviously Jeff Dean thinks it has a lot of potential and then it's you know really interesting just to think about this kind of an architecture so this is a really great article discussing a lot of interesting things relating computing and then trends in machine learning and deeper souls not to reading this paper the deep learning revolution and its implications for computer architecture and chip design you might want to familiarize yourself with this mixture of experts model and read the paper outrageously large neural networks the sparse a gated mixture of experts layer you myself might also be interested in this article on in for precision discussion around using you know lower precision for inference in this case Nvidia is experiment with using this to set the record in the ml perv inference benchmarks so definitely a lot of interesting things around reduced precision inference and kind of quantization and these kinds of ideas from Google's they had blogs discussion the latest installment of the mobile Nets the mobile net very and then the mobile net optimized for the edge TPU computing which is installed in the pixel 4 with the pixel neural core core containing an instantiation of the edge TPU architecture so mobile not virgin 3s latest installment in the mobile net family the original mobile net introducing the depth wise separable convolution for faster and more efficient computer vision models like image phonetic classification and cocoa object detection then mobile net b2 had the inverted residuals and linear bottlenecks adding this kind of structure to the feature Maps in order to make the networks more efficient then the mas net paper basically brought in this kind of reinforce and learning neural architecture search into the mobile net family and had this kind of search space described here in the paper to search for the mobile nets and then also interesting lis is the paper squeeze net which has this kind of design for feature maps the squeeze and expand where you kind of compress the feature maps and then expand them and have this kind of architecture for these faster more efficient computer vision networks so mobile members in 3 they combine the ideas of neural architecture search a lot of the recent state of the arts in order to design the next generation of mobile nets with better performance so some of this performance is the mobile net version 3 large 32% more accurate on image net classification while having 20% faster inference compared to the previous mobile net v2 so also really interesting lis to this blog post from google's AI is the importance of having the kind of hardware where neural architecture search so in the plaza in the bottom of the article they show how the mobile net edge TV you when they are doing the neural architecture search optimizing mobile net be three for deployment on the edge TPU vite devices is able to achieve this higher accuracy with you know faster inference times but then when they switch the same model into the CPUs the original mobile net v3 performs better even the mobile net be to performs better than the optimized for the edge version so it's definitely interesting to see this kind of hardware where neural architecture search for the sake of having faster models would that require less memory as well from the Rabbids a blog is a blog post on accelerating random forests by up to 45 times using the Cu ml libraries and in miscellaneous discussion on different cases of ute building rainforest and then speeding them up by using the video graphics library and putting it on the GPUs with these CUDA data frames and the CML library so the article begins by discussing random forests kind of having this idea of having different decision trees trained on different subsets of the data and then kind of these optimizations like having different ways of splitting the features and accelerating how quick you find the feature to split on for each node so say you're at this decision tree you'd first look at this feature is the size greater than and then some attribute and then you know you traverse this decision tree in this way until you reach these leaf nodes to make predictions so in the article they discuss different cases on how the C UML library is helping to accelerate this building different forests across multiple GPUs so one of the interesting things is you know building individual decision trees in this kind of random force ensemble kind of framework is an example of an embarrassingly parallel algorithm where you can easily just distribute it give it different sets of data and then build it on different you know different cores so in this case they present a example of how similar the C UML random forest API is to say the SK learn version of building a random forest and they discussed using gasp to accelerate this if you're training this on multiple GPUs so the task data frames is a little different from the CUDA data frames for distributing the you know these kinds of tabular data set workflows across different GPUs instead of just having say a single GPU so then in the article as of the benchmarks compared to SK learn and see UML so these bar charts the height of them shows the speed-up in different cases so say the you have building a hundred trees each with eight as the depth the maximum depth you would get about a 30 times speed-up in this case so in this case in this article the results presented kind of show this interesting trend where they're having a you can't as you increase the depth of the maximum depth of the random forest each decision tree you get less of a speed-up which is interesting and in the article discusses about how this will be improved in the future also interesting Lee in the article is this kind of greater than linear scaling up due to the way that the data is partitioned across multiple GPUs so this is definitely a great article really interesting discussion of rapids and the speed ups on building random forests and you can also check out this notebook from the rapids at notebook if you're interested in trying this yourself Lily and Wang posted a really great article on self supervised representation learning which surveys a lot of different self supervised learning tasks to help you to get a sense of this kind of learning hourly so from a high-level idea self supervised learning is this idea where you have a bunch of unlabeled data and you can construct some kind of task such that you can have a supervised learning algorithm from unlabeled data so a common example of this is the case of masking out words where you have a massive set of text and then you mask out words such that you can now do supervised learning to predict the mast outward so some cases of this are having this example our CNN where you construct different classes by doing data augmentation and then you kind of treat each of these as their own class so now you're doing supervised learning on the unlabeled data set by using data augmentation to form these kinds of auxiliary classes another case of this is using rotation where you take images and then you randomly rotate them such that you have this new auxilary class and you can do supervised learning so there's other cases of this like doing having the patches where you predict the relative location of the patches to each other and this is another case where you can construct a supervised learning tasks from unlabeled image data so also interesting Liam's article is discussions of different cases that kind of have self supervised learning go wrong like chromatic aberration where you can just kind of rely on the way that the colors and the image are rendered in order to do tasks like the you know relative patch location and other details like this in robotic learning how it had the model can exploit the pretext ask the social provides learning tasks in order to kind of solve it without really learning like the useful or generalizable representation of the data so in other cases this is counting features that you know discusses the math of this colorization where you you know mask out the RGB and then you predict the grayscale another interesting cases where you say just take the R and the G and the red and the green channels and then you predict end of the blue Channel from those channels it's also pretty interesting colorization tasks so other examples of this are generative modeling cases like so you crop out this patch and then you fill in the missing patch and use this to learn representations or cases like where you have the bi-directional gans such as the big bag and from deep mind where you have sort of the features and you have the encoder that takes the original dataset and encodes it into the vectors then you might use these vectors on the original data as features for you know some kind of new classification model or something like that so else in the article I discuss a lot of interesting video based so supervised learning algorithms things like tracking moving objects where you have kind of you know predicting the motion of the objects and then things like frame sequence where you kind of predict the order of the frames you know a lot of these tasks it makes a lot of sense how you would construct these self supervised tasks from unlabeled video data like frame sequence and predicting the which frame comes after which frame things like video colorization similar to the images but kind of in this case adding this idea of having a reference frame for the colors so kind of adding more structure compared to the image case and the article goes on to discuss a lot of different things about so supervised learning in robotic learning and kind of reinforced learning in these different kinds of tasks so this is definitely a really really interesting article great for you know getting a sense of the different self supervised learning algorithms that are useful and are being used specializing word embeddings for parsing by information bottleneck one the best paper award at the e/m NLP conference with a really interesting approach to compress word vectors by having this kind of variation Olano encoder and making so you say take a high dimensional word vector and then compress it into a low dimensional representation and then on the decoding end its judged based on how useful it is for doing this task of dependency parsing which you can learn about more about by going to the stanford NOP group and reading up about dependency parsers and constructing these kinds of graphs that relate to words to each other so it's definitely really interesting paper I didn't quite have too much time to get into it but it looks like a very interesting technique for you know reducing the dimensionality of word vectors pie george has published a really interesting video discussing how hi George is being used to power autonomous vehicles at Caltech so particularly the video goes through this neural Lander case where on the baseline you're not really able to model the dynamics of the drone to land it when it gets closer to the ground but by approximating it with the neural network they're able to achieve this kind of landing without just kind of like turning it off and just dropping it from a certain height so it's really interesting article video and there's also an article you can check out as well linked in the video that discusses about the use of pi torch in order to approximate these kinds of dynamics for autonomous vehicles another interesting update to deep learning and AI this week is the use of convolutional and recurrent neural to achieve this 99.6% accuracy on seizure prediction in these EEG kind of electrical activity in the brain kind of devices to extract this kind of data and then use it to train these models that can predict seizures from Stanford's AI lab is controlling assisted Robotics with learn Layton actions so to give a high-level overview of this study they have these kinds of robotic arms to facilitate tasks like taking this Tupperware out of the refrigerator and the interface requires kind of switching between modes frequently in order to control all the different degrees of freedom on this robotic arm in order to manipulate it and do these kinds of tasks so in this study they're looking to have a more low dimensional interface that can translate to the high dimensional ways of the robot arm can move because it has all these different degrees of freedom different ways that it can possibly move so they described in this paper you know their algorithm for doing this how they're trying to train these low dimensional latent spaces that can be used as the interface to control these kinds of assistive robotics from facebook's a research blog is new progress in using AI to detect harmful content so some of the ideas in this blog post are things like using self supervised learning like the xlim learning framework in order to do this kind of multilingual sort of to have this kind of detect hate speech in different languages and how you can have things like more English data or more English text data is able to improve the performance of the model in other languages as well through the use of this xlm self supervised learning framework so also in the article it discusses this more holistic approach to understanding the different ways of you know different content that they want to flag you see that they're grouping it all together into one model that has has all the different kinds of labels onto it compared to having separate models to classify each kind of offense like you know nudity or violence these kinds of things so they discuss improvements on things like detecting selling drugs on Facebook and sort of the you know increase in how good their systems are getting at improving this kind of detection and particularly the contributions enabled by self supervised learning like eczema another case is the success of self supervised learning is in Google's add blog spice self supervised pitch estimation so in this case they take the unlabeled data and then they automatically shift the pitch in different models and then they have this encoder that encodes the new sample and then sort of has this loss to make sure that it can identify how much the pitch was shifted so in this article they show house doing this kind of self supervised learning out with them is able to increase the benchmark on separating different audio files are different speakers from the same audio I think so this plot shows the differences between the spice algorithm which is self supervised compared to the CREP II which is a fully supervised model and so it's showing that the spice is able to outperform it in most cases on this task NVIDIA is open sourced a 3d deep learning research library named kaolin in pi torch so interestingly they discussed the idea of preparing a 3d model has been reduced from three hundred lines of code down in just five with the introduction of this open source library so 3d datasets and 3d applications are useful for things like 3d object recognition 3d segmentation like taking this character and then segmenting the different body parts useful for things like animation for you know rigged a character for animation or to customize the model to generate various variations of the object things like taking a two-dimensional image in and then producing a three dimensional image things like Gans or different ideas for doing this and then different other applications of three-dimensional deep learning so you can check out the library on github and video game works slash Calvin I can see some of the different applications of it and then also you can if you want to learn more about it you can check out the research paper Calvin and pie tours library for accelerating 3d deep learning research from Nvidia is a discussion on the idea of mixed precision computing so in this article they're relating the you know the numeric precision used for the pie the 3.14 one point five nine number two and then kind of relating this to illustrate this idea of different levels of precision and then also then coming back to how mixed precision is used to train deep neural networks so first to show this example of how you could represent pie at double precision single precision and then half precision and sort of showing you know the kind of what this looks like where you have less bits used to represent the different numbers so they discuss how mixed precision is where you might have two 16-bit matrices multiplied together and then having the answer be 32 bits so this is kind of the idea of mixed precision and kind of moving from different you know Precision's for having like 32-bit 16-bit floating points when training these models so they discuss how they get this three times faster when they are you know benchmarking these kinds of mixed precision systems and they discuss a lot of different use for things like earthquake simulations and then detecting weather patterns and all these different cases of where mixed precision is accelerating the training of deep neural networks so you can also check out this blog to this tutorial on automatic mixed precision for deep learning you can check out how you might use this in tensor flow with this kind of code and then how you might use this in PI torch and also I'm accident' from Nvidia is a description of Facebook's fashion plus plus system that uses Gans in order to separate texture and shape and then suggest different suggestions for outfits so it's kind of like a digital assistant and definitely interesting is this video that describes it in better detail you can see how the fashion plus plus system is suggesting these kinds of minor changes to outfits in order to improve fashion ability and you can check out more about this research by checking out the paper fashion plus plus minimal edits for outfit improvement in videos a blog discusses some of the interesting achievements by Harrison a I to improve in vitro fertilization with computer vision so some of the most interesting things about this study other than the increase in accuracy up to 93 percent accuracy or the article previously cites that this kind of prediction is usually under 50 percent and you know depending on different cases so I thought something that was really interesting about this blog post is that they don't just use single frames to predict the you know embryo development rather they're actually using these video models to have this kind of temporal modeling in their models which I thought was really interesting so you can watch this video to learn more about Harrison not AI definitely another really interesting company about you know improving this kind of medical imaging and these different kinds of applications they also discuss kind of doing this chest x-ray detection as well and kind of all these different applications of deep learning and computer vision this week's edition of the batch from deep learning Don AI begins with a note from Andrew hanging about why machine learning projects fail particularly the case of small data so examples of this are say you have 1 million labeled images and you want to detect scratch bones in a smartphone manufacturer and you wouldn't have that many examples of actual scratch phones so you might need to get good performance from a machine learning model computer vision and classifier with a hundred or even ten images for this application so this problem is also something that appears law and x-ray x-ray diagnosis medical image applications so discussing things like this kind of distribution we have this really small number of samples and you can solve this problem with things like just giving higher weight to these misclassified examples so some of the solutions these small data problems are transfer learning one or few shot learning and then encoding priors in the system with the designers of the algorithm and then data augmentation and data synthesis so definitely a really interesting discussion on small data and how to improve systems to you know achieve performance on these kinds of data sets the baton continues to discuss the case of when ubers self-driving cars killed a pedestrian so in the analysis of this system they detect how it wasn't really able to use temporal information it was switching between detecting the pedestrian as a bicycle or a car and then kind of this overall discussing what happened and sort of steps forward into how to make the deployment of self-driving cars safer and you know better tested before they're put on the roads and that goes on to discuss deep learning being used to detect fraudulent trades on the Nasdaq Stock Exchange I've been discusses a new algorithm for looking at how we fine-tune word vectors so when you take the new data set that you're fine-tuning the word vectors on kind of keeping this consistent with the original word vectors in the more general word corpus so then the batch goes on to discuss neural evolution and different algorithms such as the poet algorithm that is using neural evolution to evolve the environment and then the reinforced learning controlling agent that is used to control this bipedal Walker in this case and then also the use of neural evolution in the case of deepmind's alpha star to Starcraft 2 player so neuro evolution is definitely a really interesting use of randomized search and definitely different from you know direct gradient descent and these kinds of ideas and really interesting algorithm for the development of future advances in neural architectures the batch then goes on to discuss how computer vision is being used to count fish and in order to kind of detect their age by detecting how long the fish's which is a good proxy for their age and then how it's being used in order to control these fish populations the patch concludes with the discussion of the facebook-dot-com model so it's really interesting architecture that achieves faster inference and better performance by separating these convolutional filters into low frequency where you have just kind of like the outline the Penguins compared to high frequency where you have things like the details of the image so it's also interestingly you know the fashion plus plus system 10 does a similar thing of separating out the shape which is more low-frequency and in the texture which is more high-frequency so it's definitely an interesting innovation in the direction of faster more efficient convolutional networks and a faster inference by separating out explicitly separating out low and high frequency convolutional filters thanks for watching this edition of AI weekly update from Henry air labs please subscribe for more deep learning and AI videos