New TECH: Vision Transformer 2023 on Image Classification | AI

Key Takeaways

hello Community Vision Transformers 2023 we look at the technology at image classification task in oncology and one of my last videos I showed you the Symmetry in convolutional and N layers because of the algebraic properties of its inherent group structure in convolutional neural networks we have locality and translation equivalence inherent to the model in Transformers We lack those two features but if we train it on a large data set we have the self-attention allows us to integrate information across the entire image and our MLP layers are local and translationally equivariant so we achieve something a convolutional neural network cannot achieve because of its design great publication now with recent Transformers we have the pre-training phase where we have an MLP layer on top of our encoder stack with a hidden layer and a fine tuning where we have our Transformer encoder stack with a single linear layer currently we have 43 methods how to do a visual transform let's apply it on an application cancer screening I use here a platform from Switzerland with more than 411p reviewed scientific journals and this is the journal we're gonna have a look at today okay so let's start talking about Vision Transformer based transfer learning for mammogram classification highly interesting article from January 2023 you know what you know what I like especially about this is that here we have South Korea a National Institute of Technology Ethiopia we have Italy this is so nice of a real cooperation now they have in detail now a study about Vision Transformers compared to convolutional neural network and they have some amazing insights and this is the reason why I picked especially this publication from January 23. so they show that Vision Transformers have a better performance and they develop a transfer learning technique based on Vision Transformer to classify breast Mass mammograms muscles what is their target to improve the early diagnostic of breast cancer this is the topic we're going to talk about and this is the content of this scientific publication so here we go to tell you about breast cancer yes and then remember we look at this only from the technical perspective of a neural network nothing else I'm not a medical doctor please go see your ontologist if you have any questions we are here looking at only at the computer science level okay here we go CNN's a computationally expensive due to the multiple convolution a different features level CNN lacks the ability to handle rotation and scale invariants with no augmentation and fails to encode relative spatial information so you see they address some shortcomings in the current technology but they also tell us and this is an important message that finding good data sets for training is a challenge in a medical image domain now on the one hand I understand about the privacy and your private sphere but on the other hand if you're working in a hospital I think you should have access to the medical images that you are able to support your oncologist so here we go there are several approaches they used for the lack of training image data sets but this is not the point we are interesting here in a vision Transformer based transfer learning method for a very specific task Mama worm classification this is all we are interested in on a technical level so and to tell us when analyzing mammograms cnns concentrate on a particular region because they have inherent their local properties disregarding the rest of the image which causes the cnns to miss some crucial details which would have been discovered if the entire image was examined at once so you see this is the problem that you have with the current technology and then they tell us that the vision Transformers have recently gained prominence in the field of computer vision surpassing cnns in tasks that require natural image classification remember we are here at image classification so this is a pure classification task we know for Transformer infrastructure I showed you this when we had the classification of sentences now interesting that they tell us their vits outperformed the most advanced CNN models this is what they tell us beautiful they start a little bit about the architecture that of course they are not working with words or sub words so that they have no visual tokens but otherwise the architecture of the vit encoder stack is more or less identical you have your multi-head attention Network you have your feed forward Network run with one or two hidden layers and this is it more or less now that the market had a tension network is in charge of creating the attention Maps but not now for word or word pieces but now for the provided embedded visual tokens beautiful everything as you know it don't have to tell you any more about it yes yes yes so interesting to know is now the data set they have as always when you train a Transformer you have to have data and you have to have a lot of data and they say here we use the digital database for screening mammography and this is to train and to test division Transformer based transfer Learning System but in this database institutional database they only have not only is the reason why I tell you this 13 000 images so they have close to 6 000 images from benign and 7.1 K images for malignant tissues so remember what I told you if you want to train a vision Transformers you need millions of pictures if you really want to reach high performance so what they tell us that the databases they have with six thousand or seven thousand images of medical images is not enough so this is the reason why they explored transfer learning in this particular scientific publication yes the pictures I don't need to go here in the details interestingly for a medical publication they talk to you about division Transformer architecture and this is highly interesting because they're really excellent in the explanation if you want to have a look at this you will find it point on great they say Okay normally in the natural language processing in our NLP mod we have a one-dimensional array which is our sentence here images of core two-dimensional X and Y component and they tell you they're divided in smaller patches then they have each patch was flattened into a vector they have the mapping of the flattened patches to d-dimension using a linear protection layer and you're not going to believe it remember in bird we had the CLS the classification token or the start token now also we have here prefix and like in bird we have positional embeddings so you see more or less really the same structure only the input embedding patches is a little bit different to what we know from our tokenizer structure and they tell us the Transformer encoder Network what I call the encoder stack is a stack of L identical layers yes yes yes the MLP with a hit single hidden layer was used to implement a classification head during the pre-training so here we are in the pre-training phase as I showed you before and then when you do the fine tuning use throw away this header and you just use a single linear layer for a classification task and in this implementation you have a Galo not available yeah this is from the original publication you know this now the nice thing is transfer learning transfer learning was employed such the division Transformer models pre-trained on the large image that set this is now an image data set that has close no it has 14 million images so compared to the 13k images that they have of medical images here as you can see it is here this is now a large database where you have 14 million images so this is really where you can use this data set as a pre-training data set and then you come with your 13k medical images and do the fine tuning on top of this so this is what they tell you here please have a look yourself they say the objective was to use division Transformer Knowledge from the large natural image data circuit image net with 14 million pictures and then to classify two classes in benign and malignant tissues of course so they detach the pre-trained prediction head and replace it you're not gonna believe it with a feed forward layer where they have two classes for the downstream task this is absolute identical to what I showed you in my last 10 videos about Transformer architecture the encoder stack for bird with a decoder stack for GPT identical architectural design so this is then used for fine tuning here I told you again the weights of the imagenet pre-trained vision Transformers on this huge 14 million image data set where then utilized as the starting point like in bird you train your pre-train under huge data set so that the system learns English and then on top you do the transfer learning the fine tuning to generate a new objective function sophomore yes yes yes is all the same now the interesting is and this is amazing for a medical paper they use three state-of-the-art Vision Transformers they say hey I don't want to look at the classical vit model that I showed you before but I won't also have this window model and a pyramid Vision Transformer model so really interesting The Chosen three Vision Transformer ways methods Advanced methods and they compare it yeah this is the the window shifting architecture if you want to have a look at the detail and here down there here you have the the spatial reduction attention in this pvt pvip design of our Transformer but this is not the point just want to show you what they've done they have they said hey what about we pre-train the vision Transformer models on the mammogram data set so this is here the small 13k data set from scratch and then use this architecture and compare this from scratch from a small data set and we already know from the mathematical algebraic group construct that this will not deliver results because I showed you if you look at the CNN design of the architecture and the vision Transformer that we have complete opposite symmetries within the system and you need a lot of data sets to really achieve a high level of performance with vision Transformers so no wonder they found this argument and then they compare the transfer learning revision Transformers or the with of course a convolutional neural network implementation details yes yes yes but just jump to the result with me the proposed Vision Transformer based so the classical Vision Transformer we have here transfer learning model so not that we pre-trained it only under 13k medical images but we pre-train it under 14 million images and then we fine-tuned it on the medical for 13 K images this exhibited Superior performance so this is the winner first you pre-train it on the huge model exactly as we expected and then you fine tune it on your particular Downstream task and this was a classification in the pre-screening of an image what else this provides strong evidence that Vision Transformer based transfer learning is effective in improving the Deep learning approach to abreast mammograms thereby improving the early diagnostic techniques for detection of breast cancer beautiful our technology has a real world application that will help oncologists with their work this is exactly why we do this why we build those systems why we train those systems they should have an impact in the real world so beautiful this is more or less the result they achieved yes you can have a look at the details the bar charts and whatsoever but this is not the point of what I want to show you yet it compared it with resonant of course discussion yeah your discussion this was something I wanted to show you so the vision Transformer based transfer learning approach provided the highest quantitative and statistical measures for classifying breast mammogram fee either in a two-class system as benign or malignant issues this is an important information because if you want to know jump into the system if you want to be a medical doctor if you want to be a computer scientist in the medical field you should know where the current state of the art is what approaches are implemented what approach has the highest quantitative results and where you can start doesn't mean that the approach before is now not at all relevant maybe you can combine it you can take elements out of different methods and methodologies and form something new but it is just interesting to see the power of vision Transformer based transfer learning Approach at January 2023 that they Proclaim for a particular task classifying breast mammograms this is the superior technology when they apply it and they tell us and this is interesting I think this is amazing the prime reason for the better performance of vision Transformers is the ability to capture Global Information from the early layers of our Transformers of our encoder stack and the Deep self-attention mechanism of course that is inherent in each layer in each Transformer block that we have that enables features in each patch to be carefully analyzed for a decision making unbelievable to see this here in a publication that is focused to clearly on medical images it is absolutely in line but we would expect from Pure mathematical reasoning what else additionally they showed that training their models from scratch because the small number of images yes was not so effective we know this then they conclude therefore transfer learning provided better results as it used weights in our annual Network that were pre-trained on a large data set as I showed you on 14 million images such as imagenet and leverages the knowledge to learn from smaller data set such as here it is classical focused medical image data set of just 13 000 images for medical image classification where they then fine tune their model great publication gives us a lot of inside what's going on currently where the research is going in medical or Indy and I just wanted to show you this if you jump into this topic they're at a high level so you can apply everything I showed you in my last videos from the mathematical approach from the topological approach when we constructed specific topological spaces and Vector spaces and embeddings and structures and we talked about a Transformer architecture from GPT from bird all of this comes now here together for a medical application conclusion consequently we found that that Vision Transformer based transfer learning is effective for breast mammography image classification providing Superior performance with less computational complexity and this is really what you want you want to have less computational complexity means lower cost lower time to compute and you want to have a superior performance in the very early detection of possible my line tissues great so this Vision Transformer based transfer learning outperformed the current convolutional neural network based transfer learning four breast mammograph classification this is the result in January 2023 of this publication you have to order now it was funded by the South Korean government interesting wait and you have done all the different references this is the publication I definitely wanted to show you have a look at this and it is really outstanding because it gives you a lot of technical or computer science oriented information from their experience when they applied Vision Transformer based transfer learning for a particular Downstream task for a classification task of medical images as you can see here New Perspective to boost Vision Transformers for medical image classification it is a Hot Topic and we need more data because we do not have enough medical images available and people are trying to find new training algorithm to cope with this shortage in images if you want to see at which level the in the images are analyzed have a look at this two from word embedding and vision embedding a highly inspirational free prints you should read if you want to understand how Vision Transformers see in their particular layers addressing the shortage in available images to train look at what Adobe is doing with its cloud here we have now how many hours you have to train to achieve a performance and skip it and no this is too much so this is it for today Vision Transformers on cancer screening in 2023 I hope you enjoyed it I see you in my next video