Machine Learning and Signal Processing | Community Webinar

Key Takeaways

hi everyone my name is nathan i'm one of the marketing managers at data science dojo um today uh we're here with sarah malvar to learn about signal process machine learning for signal processing um and uh you're not really here to listen to me so i'm gonna go ahead and hand it over to sarah thanks for being here sarah hey thank you nathan hi guys how are you doing today so i want to make this a little bit interactive so please feel free to you know drop in any question in the chat or if you have any comments especially when i ask you if you feel free um to share some information that would be great okay so today we are going to talk a little bit about machine learning for signal processing especially regarding data compression and denoising and the reason is that i feel that we are kind of disconnected between you know business and academia so usually when we are dealing with machine learning and you know but for business solutions we are basically trying to solve a business issue right so machine learning is basically statistics apply to something and the point is usually we miss some of the great applications of machine learning because we feel that they are you know there's too much math involved and as a matter of fact we don't actually need to understand the whole complex or the math part uh in some cases so today we are going to talk a lot about pca which is uh principal component analysis in a way that we are not going to dive into a lot of math just a little bit but instead of using pca just as a form of dimensionality reduction when we have i don't know like a supervised learning algorithm with too many dimensions or something like that we are going to show you some of the applications of pca in rpca which is another kind of pca more robust for data compression in the noisy okay okay so first of all who am i right i'm talking and talking and you guys don't know who i am so my name is sarah mover as nathan told you uh i hold a ba an electrical engineer a master in mechanical science and a phd in engineering uh i have been in an internship in university of tokyo and university of pennsylvania working with basically with machine learning and deep learning usually i do that applied to fluid mechanics but you know my first paper was on deep learning so this is my first contact with machine learning was at the college uh many years ago so this is my core prediction i am a postdoctoral fellow at the research center for gas innovation and at the center of artificial intelligence and i have been working as a consultant for shell and ibm in deep learning solutions i'm also an instructor at data science dojo allura in awari and i am a reviewer at udacity and a mentor of mentor cruise and arc so i've been working with deep learning for the past you know six or seven years and in different applications so let's move ahead okay so just a little agenda here of what we are going to see just a little bit of introduction what is signal uh which kind of signal we are going to use how does pca work some of real-world examples of how we can use pca for data compression or denoising how can we compute this compression so sometimes when we think about compression we are just thinking about zip files or like zip folders this kind of things but there are so much more involved can we use pca to remove noise of images for example and when pca doesn't work what can we do can we use any other type of techniques we are also going to talk a little bit about other machine learning techniques and a little bit about deep learning solutions for those issues okay so when we talk about signals sometimes we think that you know what is a signal usually we think about sensors or you know something noisy but as a matter of fact we are full of signals like everywhere so if we are using a face detection algorithm we are actually working with signals so instead of dealing with structured data because usually a data scientist when he's in a company or something we do a lot with structured data right so we have like spreadsheet with i don't know tables and some information about our customers or whatever but in real life we are full of unstructured data so we deal a lot with images voices uh sensors um anything that you know it's not actually structured and we can use machine learning in all those uh different types of unstructured data so not only in phase detection but in activity identification so for example if you use any of those wearable devices like a smart watch or something it's probably recording your vital signs right now so it's getting some information in your heartbeats um how many steps per day you are walking or i don't know any other information that you allow him to take and we can use this kind of signal to do many machine learning things so first step is usually related to data extraction so if we are dealing with time domain so imagine now that i have my voice okay you have my voice is a kind of signal right when we are dealing with time domain we usually measure statistical things that we are all aware of so variance mean max the range of the signal and anything like that sometimes we have to put that in the frequency domain because it's way easier the math is way easier in the frequency domain so it can actually measure amplitudes humans the energy of that signal or anything like that and based on that we can create different models to train uh based on that input signal so we could use native base a neural network a decision tree any type of i know supervisor and supervised algorithm depending on your objective and we can infer the activity that you are doing so if i have a bunch of signals about you i can create a machine learning algorithm that is going to say that right now you are sitting watching something or studying or whatever you're not running you're not riding a bike because of the signals that i have used as an input so we have signals everywhere and i and we can use machine learning to deal with those signals in many different cases a very nice examples example is the case of the autonomous cars so in order to actually code an autonomous car it has so many sensors and all those sensors they have to be uh processed so the point is we have the visual things we have information about other cars we have information about um if we have a pedestrian or something so all those things are signals and we can use machine learning uh in order to you know actually produce an autonomous car so this is just for you to understand that we have so many signals everywhere and that we can actually use machine learning with those signals so don't think only about structured data because most of the time we are going to deal with unstructured data okay so have you know any of you heard of pca do you work with pca or have you done any pca work regarding machine learning feel free to drop something in the chat yeah yes okay this is good nope okay not yet some university projects no no okay some some knows some yes okay the point about pca is that usually in machine learning we use it to reduce the dimensionality of our issue so imagine that you have a table of structured data you have a bunch of information of your customers and you know each variable is going to be a dimension so if you have i don't know 100 variables you will have a hundred dimensions this is yes this is principal component analysis it is very difficult to work with that amount of dimensions so sometimes we use pca in order to actually remove some of those dimensions and make our problem a little bit easier but that's not the whole point of using pca we can actually use pca for compression and for the noise we're going to see first how pca works so let's suppose we have a hundred points in that 2d space okay first thing we got to do is define some of our bases so as we are dealing with the x and y like a 2d cartesian space we can define two basis function one is related to the x axis so this is represented by one and zero so remember in school we talk about vectors and vectors we have x y and if we are dealing with 3d we have the z direction right so the x direction is defined as 1 and 0. and the y direction is defined as zero and one okay so every point here can actually be determined as uh a function of those two basins so for example this guy here uh this guy is on x equals 3 and y around 2.5 okay so we can instead of just saying that 0.3 and 2.5 we can define that as a function of those two basins so we can say that this is 3 1 0 because this is the x axis and 2.5 y axes which is 0 1 okay so this is the language of linear algebra but what does it mean so basically uh i'm saying that i need both x-axis and y-axes in order to define the position of any of these points right so i i must give you information on the x and on the y in order to position each one of those data points so i cannot drop any of them if i drop the y axis i wouldn't have enough information uh for you to know where to put this specific data point however i can approximate all those points by a line right this is what we do when we are dealing with regression so linear regression for example this is exactly what we do we approximate our data points using uh an equation a linear equation that is going to uh become a line okay now let's suppose that i call that red line a new axis is going to be a x's okay and i can also create another one that is perpendicular so this is the green one that is called b x's so now instead of working with x and y i'm going to work with a and b okay and i can do that i mean i could put i could create any different x's uh in any position that i want i just have to worry that they are perpendicular okay now if we use a and b as bases we can say that most of the variance or most of the information uh of this data set these data points is along the a axis right so most of my points are almost inside that red line that b line seems almost useless right so i have created but it's not giving me much information okay so if i drop the b axis i can actually represent all my points in the data set very accurately using only the a axes so what i'm doing here is instead of dealing with two dimensions which is x and y i'm actually creating two different dimensions that are called a and b and because of the positions of a and b i'm saying that b can be dropped i don't actually need b uh for you to know the position of each one of those points okay so in pca usually that's a mapping mathematical concept we have four steps so the first step is find a new set of basis functions and of course don't worry about it usually when we are coding uh and i'm going to show you that when we are coding this is just you know one line of code we don't actually have to implement that but just for you to understand what is happening behind it um so first thing is to find the new set of basis functions where some of the x's are going to contribute for the most part of the importance in the data set the second step is to arrange these x's in decreasing order of variance contributions so in the previous example i only have two bases i would say that a um is the most uh so it's the basis with most information so that would be in the first position and b would be in the second position because uh smallest variance contribution we have step three so i have to pick the top k axes to be used and drop the remaining ones in the previous example i would get only the a axes and i will drop the b axes so it doesn't matter for me about that green line and on step four uh i can project this new data set onto this onto these uh key axis so that means and now i'm going to enter in the only math part that you have to deal with um when we are dealing with pca basically what we are doing is mapping something that is in a higher dimensional space into something that has less dimension so this is why we call uh dimensionality reduction technique and in linear algebra we have two consoles concepts they are called eigenvectors and eigenvalues so the eigenvector is exactly those new bases that i created so i have the x is one or a and the eighth is true or b and i have exactly the their position right and my agent value is a scholar uh that i'm going to multiply my agent vectors and it's going to give me the amount of information that that eigenvector is actually giving me in order to calculate a contribution so if some of you have worked with pca usually we have that plot uh it's similar to that elbow method in clustering analysis we have that saturated curve that explains the amount of contribution depending on the number of dimensions that we are choosing so this is how we calculate we just get the eigenvalue of determine basis and we divide by the summation of that so in this specific case if we calculate that we find that 99 almost 100 of my information uh is related to this new base is called a so the a axis while only 0.3 is related to the b axis okay so the importance of every eigenvector towards explaining the information in a matrix is directly related to how big its eigenvalue is so my eigenvalue of a is way bigger than my eigenvalue of b that means that i'm having a lot more information in a that means that i can drop b so i begin i have begun with um a two-dimensional problem i have data points in x and y and now i only have one dimension that is a so this is the whole idea of pca trying to reduce the number of dimensions that we are using and this is very important for clustering analysis for example so let's suppose that you have again a bunch of information about your customers and you have i don't know like 50 variables sometimes they are not all necessary in order to conduct an unsupervised machine learning technique we can actually try to decrease that in order to use only the information that is definitely uh necessary okay so again what if i drop my b axes and i want to reconstruct my data only using the a axis this is my result so we can see that i will have only data points that are within the line so they are on top of the line of a line and that means that i'm going to lose uh around 0.3 percent of information okay so i lose uh almost known information and now you might be wondering okay so what is actually happening i cannot see that so here's an example of something that was 3d with a bunch of positions like a thousand positions and i have conducted a pca and now i have a 2d uh space with 100 positions and i can also conduct a pca again and transform that into one dimensional with 10 positions and every time that i do that that i reduce my dimensionality i lose a little bit of information so when we are dealing with compression uh we have to keep that in mind i am losing information so if something is very necessary uh use pca with caution okay let's go to a real world example so where can i use pca you know okay you gave me the customer example and whatever but is there any other nice way where can i can use pca to compress some images for example this is uh the example of the paper called again faces so this is basically a data set with many human faces okay so 13 000 human faces and they're all in they are all grayscale images and they are 64 64 pixels okay so i could use that data set in you know as a as a whole but i have to of course transform those 64 per 64 into an array of pixels that would give me so much work so what i can do is basically try to understand the features so for example each photograph we have some facial features that are present and all of them like eyes nose mouth right so instead of using the whole information pixel by pixel for each one of the 13 000 images that i have i can create templates and if i combine these templates i can generate any face in the data set so i will create smaller templates with simple information and i will use those templates as basing to create new images or to reconstruct the images that i have okay so each template of course uh is going to be 64.64 so five four thousand dimensional and i can reuse these templates as basis function to generate new faces in the data set let's see how we're going to do that so first step i get uh those 5 000 dimensional space that will be generated and each phase in the data set can be represented as a linear combination of these templates okay so again i have a bunch of faces i will get some of them that are going to be my basis functions and i'm going to recreate any other image that might appear in my data set using only those templates okay in step two i'm going to arrange these in a decreasing order of variance contributions so i can drop the templates that do not provide much information the same way that we have dropped b axes and on step three uh we of course if we call that we found that to retain 99 of the variance or of the information we only need the top 500 templates almost 600 templates so this is one uh this is those are examples of templates we you can see that they are not faces right we can see a nose we can see eyes we can see mouths but they are not exactly faces so this is the idea we are not actually computing faces we are computing templates that are um linear combinations right this is how we compress our data so now after that we can construct a projection matrix and we can project the images from the original 5 000 dimensions that were the 64 per 64 pixels uh using only 600 dimensions so for from 5 000 to almost 600 dimension that is a major thing right and now how can we actually compute that compression so initially we had around like 30 000 images of 64 per 64 pixels so a bunch of unique values we can actually start to remove the values that we have used to create the templates so after pca we use only almost 600 constants for each image so instead of using those 4 000 we come to 600 constants for each image so those are my new unique values i also have to store the templates that i use to reconstruct the images so the templates are those strange images that don't actually look like faces but those are my basis function this is how i actually construct new faces so i also have to store them and as i compute that i realize that i have 81.5 of compression so instead of you know using something to storage 13 000 images i can store it a lot less now and i of course i'm going to use a lot less memory i'm going to use less computational cost so everything is better when we use compression okay what about the reconstruction so okay i use those templates as basis function in order and those templates remember the strange images that don't quite look as faces but how about the reconstructed image because remember when we have done that example using the points some of the points were missing after we have dropped the b axes right so now i'm probably going to have the same problem when we deal with images right yes we are going to lose a little bit of information so as i told you pca is something that is not a lossless technique so we are going to lose a little bit of information and in the images it becomes a little bit of blurry right so we can see that the images are a little blurry but you know the idea and the image itself is right here after we have compressed that in 81 so major advance right what about noise um we have seen that pca is removing some information that might not be so necessary right so if i'm thinking that noise is a type of information that is not necessary maybe i could use pca to remove that kind of noise and i can do that with signals like voice but i can also do that with image so any component with the variance that is much larger than the effect of the noise should be relatively unaffected by the noise that means that any complement in in a specific image that has a bunch of information it's probably not going to be affected by any type of gaussian noise so in that example we have a handwritten digits that were digitalized and we have added a little bit of noise and of course i cannot use that as an input in any machine learning algorithm it's not so my results are not going to be good it's probably going to affect my metrics right so i can use pca to remove this kind of noise so after i remove some of my information of some of my dimensions i can actually uh have something that doesn't necessarily present so much noise so in this case 50 of variance amounts to 12 principal components so this the first 12 principal components explain 50 of my images of course this is not perfect um and any noise is a problem in any type of data but this is a very good way to remove noise especially when we are dealing with images this is something that we are now doing in xiao uh and it's showing some promising results so pca is also a useful to feature selection routine why because we are getting the most important information so instead of dealing with i don't know like 50 um 100 dimensions or variables we can actually decrease the amount of variables that we are using only getting the ones that are more important okay so rather than training a classifier uh on a very high dimensional data you could apply pca and try to use that same classifier on a lower dimensional representation remember every time we have a problem uh we want to make it the more we want to make it simple we don't want to add complexity that is not necessary so if we have something that could be simplified this is always the best alternative but pca doesn't work well in some cases and this is where the math becomes you know kind of complex so i'm not going to dig into the math but just for you to know uh when pca won't work sometimes our data is sparse right so imagine that you have i don't know like a matrix again customer information but you have many categorical variables and you have transformed them into zeros and ones we are going to have a sparse matrix right so this is a big matrix with a bunch of zeros and a bunch of ones pca is not going to work well on that this is just a plot of so in that case if i change um my bases from the black one to the red one nothing is going to change much in that case on the b case things are going to be better so this is just a graphical representation of when pca don't work this is the a cases uh and when the b and when it works this is the b case okay uh and another example where pca won't work is when we have uh a non-convex space so when we are dealing basically machine learning is just trying to optimize things right so uh when we are trying to optimize things we have a space with our function and we want to find the global minimum but sometimes we have local minimum that means that things are going to get stuck there and we won't be able to go further or to improve our metrics this happens when we have a very complex problem right a physical problem or something like that uh in order to say if something is convex or not this is just a very simple example so if i draw if i draw a line and my function is not shown below that line i can say that this is convex and if it shows below that line it's non-convex and when i have a non-convex problem i'm going to have some issues if i work with pca okay uh so okay city's asking isn't a feature space with large number of dimensions parts by definition it depends uh if you don't have a bunch of zeros for example it could not be sparse actually yes some data sets are just huge that's what samantha said so for example if i have a bunch of i don't know like let's see an example here sensor values of a machine i don't know you are measuring a bunch of things in a machine uh of course we are going to have a huge uh data set but the values are not going to be this is not a sparse matrix because you know you are going to have actual actual values that are giving you some information about it so it's it's quite different do the aces for pca have to be orthogonal um well it depends usually when we are dealing with bases when uh we are mapping things especially on cartesian space we are dealing with things that are orthogonal uh by definition but it depends on the space that you are working on okay uh how would i know if my training image is high dimensional data as i'm working with several steel data sets okay so your images are probably going to be high dimensional if you have a bunch of pixels so look at the memory if you are dealing with um i don't know eight by eight image like eight pixels by eight pixels uh well you can compress it you can always compress things but you know uh it's not much of a problem but if you are working with an image with high definition you are definitely going to have a problem about about dimensionality what about pca with time series data um yeah this is this is a good question usually when we are dealing with time series especially which uh i'm thinking that you were talking about one variable that is time dependent right um if we conduct pca we might lose information and that might change the autocorrelation um between the observed values right so this will definitely affect any forecasting that you would do i wouldn't recommend doing anything like that especially because it is already a two-dimensional information right you have your x-value and you have sorry you have your y value and you have the time so why would you conduct pca right okay let's go ahead um when we cannot use pca we can actually use robust pca so again i'm not digging into the math because it's a little more complex but basically we are trying to separate our data into a low rank feature component and i spar strong background noise so we use a lot we use rpca a lot when we are trying to deny something because noise by definition um it's an information that you know it's usually sparse sometimes we have a noise that is well um spaced for for example gaussian noise usually our pca doesn't work quite well with that because it is um it is spread right into the information of your image but if you if you are dealing with salt and pepper or speckle noise something that is like a corruption that would be a great option to use rpca so we can actually separate the low rank which is like the basis and the sparse matrix where we are going to find our noise okay however we can also use our pca with faces again this is a very nice work by zoo uh where we have like the original faces and he's trying to remove the shadow so the shadow will be the sparse information so the low rank information is actually the faces and the sparse is exactly the light or the shadow on top of that information so it's a way to separate information instead of removing information as we did in pca with our pca we are actually trying to separate things okay this is another example of how can we use our pca again in images it works so well with faces that people have been doing that for the past few years so we have the original faces and you know the guy is smiling the guy is using some some glasses and so on so the low rank this is the base information is going to be his face and we are just adding a little bit of information that usually um is something that is a little off so i could say that this is an outlier it's something that is not in the basis function so the sparse matrix as i separate things on the low rank matrix and on the sparse matrix would be the glasses or the smile something that is different an outlier a noise some information that is not the main definition of the data that i'm using okay so this is a quite good example of rpca and i wouldn't recommend using pca for denoising i would definitely go for rpca in almost every case because it's very hard to find a convex problem and again you have a difficult time defining how many dimensions you should remove so go for rpca and you can actually define the amount of separation that you want to do would you like to uh how would you build your space matrix okay this is another example using uh experimental data so we have just a flow and a cylinder this is a very common fluid dynamics problem and depending on how we separate my low rank and my sparse matrix i can remove data i can remove noise perfectly or i can actually leave a little bit of noise uh in the low rank and i can actually just you know throw everything to the sparse matrix so this lambda here which is related to the lagrangian multiplier that's why i'm not digging into the math about it but the amount of uh information that you want to put in your sparse matrix this is how you define if you're going to uh correctly separate your noise or not okay uh okay so here again another example we have noise on the whole uh problem on the whole dimension and depending on how much we separate the lowering and the sparse matrix we can actually have good or bad or not so good results of course it depends on what you want to do right would you like to remove noise uh or if anyway whatever it depends on your objectives but you can separate information and we can actually use other machine learning techniques so it's not only pca right we can use uh for example k means this very common clustering algorithm uh to compress data so we can reduce the color components of an image for example depending on the amount of clusters so remember k means we have to previously define the number of clusters so in the case of i don't know let's suppose i'm dealing with image with a bunch of colors right and i want to compress that by removing some of the colors that are not that important uh i can define k as the number of color components that i want to preserve in that image so instead of defining k as the number of clusters i'm going to say that this is the number of color components that i want to keep on my image because you know sometimes we just look at an image and there's no difference between a specific type of loop with blue and a specific type of any other blue right so we cannot see much of a difference but this is a way to compress data so remember again we are losing information so if you cannot lose that information i wouldn't recommend this kind of uh algorithms okay as we increase the value of k so as we increase the number of clusters we will also increase increase the number of colors and we are we will get uh closer to the original image so if we have disk space if we can afford use all that memory that's fine but usually memory is a problem usually computational cost is another problem so this is a way to make your data more fit and we can also use some deep learning solutions those are probably like the most common when we are dealing with images especially because uh classical learning algorithms they require feature engineering so we have to make all those steps before adding an information to a model so we can define which variables are important or not and all those kind of things when we are dealing with deep learning we don't need to you know previously define which features are going to be important or not because the features are learned through optimization right and the deep learning scales um much the deep learning scales much better with more data than classical learning algorithms algorithms so if i have a bunch of data like one of you asked me about what is a high dimensional image like if i have high resolution images i would definitely go for deep learning instead of classical machine learning algorithms okay and the accuracies of deep learning are usually also way better when we are dealing with signals that you know they are not structured data so switch processing natural language processing computer vision if you want to make something related to facial recognition so computer vision it is you are definitely going to work with deep learning and of course reinforcement learning we cannot deal with reinforcement learning without uh dealing with neurometrics okay and there are some examples of how we can use specific types of deep learning solutions especially for the noising or data compression so in the case of generative adversarial networks guns we basically compute two different networks so they basically train and compete against each other and that usually makes our accuracy very good so i have the encoder that is going to encrypt the information of an image for example and it's going to compress my image so by definition i'm going to compress my image so it can uh be something that i i'm getting information from and after that i have the decoder or the generator this is going to use my compressed image uh according to the information provided and uncompress it okay so for example this is a neural network that uh was created by young so this is actually a convolutional neurometric and it we have basically two networks one of them will take an image and will create a compact representation okay so i have this butterfly image and i'm going to add a bunch of different layers and they're going to compress my image and create a compact representation after that i can use any type of codec like jpeg jpeg is a way of compressing image right uh and that will be passed to the second network so the second network is going to fix the image from the codec and try to get back the original image why is that different from doing pca because in pca we were getting something a signal or an image and we are basically losing information so when we drop a basis when we say okay i don't want that b axis i'm losing a bunch of information and i cannot reconstruct that i cannot obtain that information back so when i'm dealing with this kind of deep learning solution i am actually trying to store that information or sometimes not store that like in that case i'm actually compressing the information but i have another network that was trained with that first network and i can create an upscale image and i can reconstruct my original image so this is a way of dealing with information without losing it so it's a lossless um compression technique and it's very useful you might want to check on github there are many different um solutions for increasing the resolution of an image of course usually when we are doing that we are you know trying to create just a basically a interpolation right so if i have an image with have you seen csi right so they get that awful images of the i don't know the car or something like that and they just zoom it and it suddenly became a high resolution image right so one solution for that is using deep learning so it's basically going to interpolate data and try to get information that was missing because of compression for example so this is a very good example of how we can deal with image to compress decompress and anything like that uh so i i have a question here i heard that convolutional encoders will help in the noise as well how will this differentiate with other methods here yes so this is a good question jason so there are many deep learning solutions for the noisy and i would say that they usually work quite well the main result is pretty good and of course if you are dealing with noise this is kind of easy to say because you can just get the information without noise and with the noise and you can use a bunch of metrics like similarity index or something like that l2 norm in order to compute if you have correctly removed that noise but the the main critic about deep learning solutions is that sometimes our networks are not really um understanding the physics they're just understanding the patterns but if you have not given them enough pattern they won't they they wouldn't know you know if i don't give an information to my model it's not it's it will never know how things would behave with that information right so but yeah it's it's a good uh it's a good question so now you're open to questions and answers so if you have any question uh and i cannot see the q and a tab only the chat okay so i'm going to answer the chat questions because i cannot look at the q a but what machine learning method would you use to learn from turbulent systems great question see so the problem about turbulence is that well it's basically a chaotic uh physics behind it right we don't actually know what's what's happening but there are many ways people usually do uh dmd so dmd is basically uh a type of pca we are just trying to get is the compositional mode right so we are trying to get the modes that are giving me the most information and as we are dealing with turbulent flows we are working with uh control so we have the positive poles and the negative poles that are going to maintain my systems table or not so dmd is a good technique to you know try to get the information that is actually relevant so i would search for that uh if you want this dmd okay so how we can determine number of x's i have data set with 122 features where what number of pca x's okay so let me show you a quick example right here uh i'm going to paste that on on the chat this is a jupiter notebook from that um very good handbook python data science handbook and it has a bunch of information on pca but usually what we do is we plot that graph so this is the cumulative explanative variance as a function of the number of components so i have a bunch of components right so in your case i have 122 components and i'm going to plot how much information i get as i if i use 10 components for example or 20 or 30 or 40. and usually when we get something around 80 percent 85 90 of course it depends on your problem but anything above or at above 80 or when the curve actually starts to saturate this is a good indicative that your model would be well explained by in this case like 20 components so this is the kind of plots that you would like to to plot okay is it possible to apply machine learning to a data set with only three raw features for multi-class problems uh yes it is it depends on what you want to do right so if you want to i don't know like classify this stain based on three uh raw features that's a a very good example of that with this iris data set so it's a bunch of flowers and we have the the the different uh sizes of the flower and i believe we have only like three features or something like that and it basically uses a classification algorithm to segmentate that but of course it depends on your business problem if that's what you are trying to do classify you can actually use three raw features no problem okay how much variance do we need while doing pca so again um for this normal type of business problem anything above like 85 or 90 would be a good uh approach but it depends if you are working with images sometimes 95 99 would be a good choice because you know we have just compressed things on 81 and we still have a good result but sometimes we lose too much information and we cannot get it back so be careful with that working with categorical features what's the best technique for converting this category features into numerical uh one hot encoded so you can just be of course you're you will increase the dimensionality so again if you have like gender sex uh whatever female and male you're going we using one hot encoder you are going to create two new columns or two new features for the data you show with data points along two lines separated by about 20 degrees could use true non-orthogonal is for the bases in that case uh i wouldn't recommend if you are dealing with a cartesian space if you are not if you are working with any other type uh we can but again this is in my case i have used only two dimensions right if we have many other dimensions we are probably going to have something that is non orthogonal is how out to encoding neural network in some sense similar to pca um [Music] well not exactly because on pca we are removing the infrared so we get what is what is most important but we remove the rest we throw it away on the outer encoding we don't throw it away we compress it but we have the features that um and this is this answer is only if we also have a decoder right so if you have a decoder that is going to get that information the decoder was trained to gather with the encoder in that case we won't lose information that is the example that i gave you however if we are not dealing with encoder and decoder we might lose some information so in that case it will be similar to pca yeah i heard that convolutional motor encoders could help in the noisy i think i have answered that about pca in time series if i'm monitoring any variables in a time period i can apply pca taking the entire array of variables at the same time t as a single n-dimensional point to reduce does it make sense for anomaly detection i wouldn't go with bck for anomaly detection there are many other algorithms uh svm or even a neural network that can actually detect anomaly so i would take that time series and use an unsupervised or even a supervised learning algorithm to check anomaly okay so i wouldn't use pca for that uh okay collab thanks for the session when missing value is more than 50 of the selected features can we drop it or make it imputation what imputation techniques well there are a bunch of implementation techniques we can do you know median mean value zero it depends on what you you want to do uh is there a way to know if my model for some prediction is the most optional as can be look at the metrics and uh it depends on the problem that will differ the types of metrics that we are dealing with what is the best deep learning models we can use for prediction of time series data set lstm so it's a neural network that can actually add memory to it this is the deep learning model if we are dealing with machine learning only arima usually works quite well and do jpeg image encoders take into account if there's a neural network at the end of the decoder uh no they don't so when i'm dealing and it i mean they use jpeg but they could have used any other type of encoder so it's it's not actually related they just use it uh as a as a tool right it's not actually related to the neural network uh okay if i would like to detect still the facts and image is it means that i should create templates of the defects here uh this is a good example jason so if you um if you have too many images and they are like high dimensional and you don't have the computational uh i mean if the computational cost is too high you can actually create templates of the defects this is a a good solution that you found i would definitely check this um again faces paper because i think that might be useful for you if you have much data yeah why not and if the the facts they have uh you know specific information like if they repeat somehow yeah that that's definitely something and last question uh pca is still viable if the relationship is weak between variables yes it is because when we do pca uh we are actually create so let's suppose you have i don't know like 50 um 50 dimensions and you go by 10 dimensions those 10 dimensions are actually a linear combination of the first 50 dimensions so what's going to happen is that the coefficient is going to be small in some of the features but as a matter of fact you can use regardless of the correlation between the variables all right sarah i'm gonna jump in and stop you there um thanks everyone for joining i wish we could have sarah all day for you to answer your questions but um unfortunately we have to go now um i did just a comment feel free to send me an email or find me on linkedin if uh you have any further questions i'll be more than happy to answer that those are my my contacts and you can find me there all right thanks sarah um i d we do have another webinar march 10th with zia hydaria assistant professor at the university of pittsburgh his uh presentation title is miles to go before i sleep gamification health wearables and health activity he's going to be presenting his research on the effectiveness and benefits of fitbit of leaderboards and uh competition and gamification should be really interesting um i've posted the link already to the details once i'll post it one more time thank you all for joining we will be posting this on youtube it's all been recorded and we hope to see you on march 10th have a good rest of your days bye bye thank you