Fraud Detection with Graphs

Key Takeaways

[Music] you're listening to data skeptic graphs and networks the podcast exploring how the graph data structure has an impact in science Industry and elsewhere welcome to another installment of data skeptic graphs and networks today we're taking on a topic I've been eager to take on hopefully there'll be even more on this but today it's all about cyber security and how networks can be a tool for finding fraud and crime and other malicious activity Assaf is this an area you've worked in before yes and uh I have some uh bad news and good news about it so people often turn to me and say well we can use Network science and networks to do some anomaly detection look for suspicious actions in the organizational it or network and find out if there's something malicious is going on but if I can plug the Show's name I'm a bit skeptical usually when people um to to cyber defense Solutions you know they uh want to get the alerts the first demand that a customer has a client has about cyber security alerts is don't give me all the alerts right give me it by order give me a least a list of alerts usually I guess it's his second demand is how to turn off the alerts right you get a a long list and when you you dive into the first one and try to figure out what's going on on what's going on there all the while the alerts keep accumulating the good news is that if you transform this list into a network like let's say an IP network right which IP connects to which IP and you look at the at the problems or the alerts this way the network can easily Focus you on the main problems right the ones let's say with the degree or and you can see which problems are a subset of others of other problems right because it's a subgraph in the community graph this is my take about cyber security yeah he uses this nice technique uh hmil the Hier artical multi-instance learning where instead of having you know somewhat traditional data set of everything's labeled you have a bag of uh instances and those share a label through that technique I guess is what how they achieve scalability that their graph neural networks weren't scaling well for this particular problem set and that technique was which I found pretty interesting to the best of your knowledge is there a lot of graph and network effort going on in cyber security it seems it'd be a focus in that industry I know I uh advocate for Network science every everywhere in fraud detection also and that's why I uh started the Netflix the network science podcast to to show the applications that networks could have in each in every industry actually absolutely Shimon has a unique opportunity here he's in partnership with Cisco so they have a bunch of low-level Network routing data that I'm sure is very insightful I wish I could have got my hands on that I'm a little jealous of the project he got to work on here well let's jump right into the interview let's um Shimon mandle from chick Technical University I also work at research La in Jen former aast and I'm pursuing my PhD at at the University at the same time in my PhD I'm focusing mainly on on Json data what we would like to do is to have some kind of a set of offthe shelf algorithms that we have for images that we have for sequences that we have for texts in general we would like to have this also for Json data this type of data is quite widespread I would say but still it's quite ignored by the by the machine Learning Community by the research community so we we like at least during my PhD I want to fix that well one of the distinguishing properties about Jason data to me is that it's inherently unstructured there's not typing there's not a heavy schema to it that could be its strength or its weakness how do you see it one one uh constraint that we or assumption that we work with is actually that this kind of data has a has a schema and we Define it in our papers and one of the first things it's import this schema is important it's quite Loosely defined say they can that there can be missing data you can you can have several data types at the same path stuff like that but still you have to have something like that otherwise it's very very difficult recently with the rise of large language models you could attempt to uh to do something like ignore schema Al together and push everything through through the LM and see what what it tells you but it also has some problems so one of them is the context length other is the computational complexity of Transformers so it's not that straightforward ad Json is a hierarchical data structure which consists of atomic data which composes let's say more higher level structures objects that represent some objects in the real world the schema as we Define it is a I would describe it as a hierarchical data type so let's say you have a Json that describes people each person that is described in this Json can be a list of people for example it they may have an H which will be an integer they have a name which is a string this is how we Define the schema and once you have it you can build a model that is able to process these uh these kind of structures even though we want the schema to be fixed for one particular data set like the framework that we research or that we propose in our papers it can handle basically any any any schema given one particular data set you need this data set to have some fixed schema but when you switch to another data set uh you can you can have a different schema Ah that's interesting and then is it that the techniques are Universal or the techniques can do an interchange between schemas techniques are you are Universal so you you you can reuse a lot of lot of things between different data sets obviously uh some data sets are more difficult than others so some are some are very easy some some data sets so you can just use default stay and get a really really good accuracy so what are some of the domains you can apply the techniques in at the beginning when I said that this this is quite it's it's a very specific and you could say Niche Niche kind of data we see it a lot in cyber security so one interesting application is that when you want to learn something about an executable say you can do static analysis you can run it in sandbox uh and get some information this is what we call Dynamic analysis what is interesting is that all the tools in this space that provide you this they are called for example file info for static or cuckoo for dynamic or gvma these are desent books is the names of them for each executable they output Jason this is one one big source of this this kind of data what the cyers community usually does is that they take this Json and then they like think very hard about what to what to do next with that they want to apply some classifier say tell the difference between malware and benign files and they but they if want to apply uh some of the Shelf standard classifiers like neural Nets or decision trees or random Forest they need to have a feature Vector which is a fixed size fixed length array or numbers say or array of categorical variables but it's only it's only one dimension this is this is a really hard constraint on these models what researchers do in this in this era usually is that they Define this mapping from J to features and these can be really really high level really powerful features but the problem is that this needs a lot of knowledge a lot of expertise in in this domain and the other problem is that these features can get obsolete very quickly especially in cyers SEC which is a very Dynamic uh landscape very Dynamic domain one example here is the cyers SEC and what you can also do is to take our framework our framework which is called Json grinder JL written in Julia language and you can apply it on on this data and you can pretty much out of the box it's like 20 lines of code and you get a pretty good Baseline without anything that I've just described we also have not only these out these outputs from sandboxes in cyers SEC also the paper that we are discussing today is another another example of nice application of uh of of our HML framework so let me see if I've got the cyber security example right you've got a suspected piece of malware and there's maybe these common sets of diagnostic tools that are run to sandbox and see if it does something suspicious but what I'm picturing coming out of that is probably pretty raw data logs and stuff I I don't I don't imagine the machine learning is at that layer that there're these one size fits-all tools what does the raw data look like and how do you turn it into a feature Vector in today's digital age the sheer volume of personal information scattered across the internet can be daunting I've been there that's why I was intrigued by delete me's approach to digital privacy what sets them apart is their flexible user Centric process that puts you in control one of delete me's standout features is their customizable privacy protection when you sign up you decide exactly how much information you want to protect start small if you prefer then expand your protection as you witness their effectiveness firsthand through detailed removal reports their service goes beyond onetime removals delete me actively monitors and eliminates any new or recurring data throughout your subscription period their team of privacy experts handles the complex removal process with hundreds of data Brokers making digital privacy protection effortless for you experience Peace of Mind knowing your online privacy is in capable hands visit delete me today and take control of your digital footprint keep your private life private by signing up for delete me now at a special discount for our listeners today get 20% off your delete me plan by texting data to 6400 that's data data to 6400 message and data rates may apply what does the raw data look like and how do you turn it into feature Vector the point is that you don't that's the main selling point of the HML HML stands for by the way a hierarchical multi- instance learning you had to turn these these logs these uh these they are very rich some examples when you run some binaries through cuckoo you can get some files that are T megabytes of data so it's really it's large yeah with the current approaches you would have to do something like this you would have to transform it into a feature Vector but uh what we do is something a little bit different we just read the file we represented we infer the schema we we get a data set we read 100 100 samples and we infer the schema so that we know these Json they have here and when we're talking about the cucko cuckoo sandbox we get for example opened files during the D execution we get where the file wres we get what CIS calls the the executable does so everything like that we can get into our schema and then load the document into an inmemory hierarchical structure and what follows is special hierarchical model which is based on multi-instance learning there are some projections with a couple of neural net layers there are some aggregations and we basically have some sub models it's it's all very hierarchical and recursive so we load the we load the document into the memory and process it as such and we get one vector as an output which you can consider I don't know as an embedding and you can slap another another classifier after that and train it so it is another another term that we use in our papers is that like H Mill is basically a a learnable embedding for Json files or other similarly Hier ially structured data so you've ingested it you've inferred the schema you have essentially structured data at this point um how do we get to an embedding is it through the traditional techniques or is there something new about it this is the multiple instance learning part so multiple instance learning is quite old already an idea in the very beginning multiple instance learning was was an extension on standard machine learning so in standard machine handing learning you have this one fixed size vector and in multiple instance learning you get a set of these vectors you get a set of these vectors and you get one label for this set of vectors the set of vectors is called a back it's called a b and you want to you want to predict one label per back so instead of a data set that consists of collection of vectors you get a collection of sets of vectors and what multiple instance learning does is is that it it researches models for this type of data but as I said this is quite already quite an old idea and uh it's been since extended to hierarchical machine hierarchical multi-instance learning you not only can have a back of vectors or a set of vectors you can have a back of backs of vectors you know you can recursively compose these Concepts together essentially we got to we got to our Json because these are nothing else right is it the case that the bag is known to all share a label or is it more like voting where you're picking the most appropriate label given the population in the bag and it it depends on the data set so in early Works in this space they sometimes assumed that yeah each and I didn't mention that these vectors in the back or these elements in the back because in hierarchical multi- instance learning it doesn't have to be a vector it can be something more uh more complex these are called instances so the elements of the B are called instances and in early works it was assumed that each instance also has a label but you you don't you do not observe these labels you only observe the label of the whole back nowadays it doesn't really matter you know because we have neural Nets you can approximate any any any function it doesn't matter whether there is this assumption or not so you can for example uh when you have a list of people each each person can be classified into some category and then they can vot together to get the label yeah that this could be some underlying function that provides the label that we want to approximate yeah it could could be part of it yeah very possibly and also uh the same could could could could hold for the sandboxes that I talked about so you could classify CIS calls so this these calls to to the core Os Os functionalities you could classify each of them uh with some severity whether if it's very very suspicious or not you could then compose the global label out of these sublabels let's say yeah it's very very very uh likely something that is happening under the hoood when we training the models and another part of our models and also what multi instance learning does is that it needs to somehow aggregate this information aggregate it so we have we need to design aggregate aggregation functions most often than more often than not just mean or Max is is enough and here you can you this is basically what you describe with with uh with getting one Global label out of out of several lesser labels yeah well I'm I've been led to believe that in cyber security one of the most useful types of features is a frequency counter so how many times has this person failed their password in the last 90 days is zero but they failed it a 100 times today that's pretty suspicious so I know features like that are very present in cyber security where they're just counting how often something happens or have they seen you from this location before does your vector sit next to data like that in a deployment or do you kind of subsume and automatically figure out frequency type features the model can definitely learn to count something if you have an array of things in your Json or in your document it is very very easy for the model to learn to count these things practically speaking when there are a lot of these things and you just need the counter just the frequency it's can be quite wasteful honestly like if you if uh If You observe thousands of events per day and you need to just remember their number if it's really an Overkill to train a machine machine learning model to do that in all of machine learning and in cyers SEC especially it's it's all about uh it's all about some trade-offs so you have you have some domain knowledge you know which works which uh which things work which uh things don't so you put some some of this inductive bias into your models this is what feature engineering does and uh there's a wide spectrum of what you can do from really taking your time and effort to design a powerful high level features but this is this is very expensive to just collecting the data and uh letting uh letting machine learning do its work so we always need to decide where we want to stay and basically what we do with the HML framework is to provide another point of the spectrum so it's less uh let's let's less manual labor intensive than designing the features but you still get some of the machine learning magic that you that we really like yeah so through this process you've um I believe the output then is these vectors where you have a really nice embedding that describes the data considering its hierarchy and its structure and all these sorts of things what's your next stage or maybe we should jump ahead and say what's the objective what are we trying to infer from the data yeah it depends so as I can as I said uh you can regard any HML model as a learnable embedding and you can do a lot of lot of things with the embedding so you can glue a couple of neural net layers at the end and learn some classifier you can train not only classification but also regression you can consider these embeddings as some latent space depending on which loss function you select you can also train a very nice latent latent space model it's quite similar I would say to what what standard machine learning does for images we have some latent spaces that you train you can classify images you can generate images so we we can do all of that basically HML H framework is just a step from a document to to this vector so how do graphs play a role in the process one uh obvious thing is that you can consider each Json document or not only Json but also XML and other other types of these documents you can the these are trees basically from from graph point of view this this paper that we're talking about it's about very nice application of H Mill to graph data what we did is that we get some data from from Cisco this data these were graphs basically and what you can usually do with graphs is that you can you can also apply machine learning there is a rich rich array of of methods for that as well um one of the most prominent are graph neural Nets and and models like that we tried a slightly different approach um and we had several several reasons for that the the first one is that the the data was large like like uh exceptionally large we couldn't even dream of applying any kind of this uh these graph neural Nets which are really powerful but are better suited for really smaller types of data if I should describe this this kind of data it's basically just a very very simple observations and even you can you can see it as binary relations so you get you get a set of clients a set of computers in a network and you get a set of domains let's say second level domains so these are the domains that the computers the clients connect to and you can collect data at the at the at the edge of the network of the local network and you can just write down uh to which domain which which client connected so you you will you will get these pairs and you can also uh not only observe clients you can also observe binaries so executables you can write down this executable you can write down uh some hash of the executable and you can remember that so at the end of the day you are left with a really really long list of these connections or of these informations it's just binary relations just a just pairs from this you can model higher level patterns of behavior of these of these uh of these objects in the network you can observe that the two two clients connect to the same same same domain Etc you can already derive rules like that and this leads to to representation as a graph you can transform this into a graph and this graph is still very very very very huge classical standard graph neural Nets didn't can't really can't really be applied here so what we did is that we we just applied HL HML here or the HML net architecture yeah the difference between graph neural Nets is that graph neural Nets they apply message passing over the whole whole graph whereas hm net just views the graph as a kind of database so what we describe in the paper is that you we let's say we are interested in one vertex in the graph which represents one client for example or one domain that's better one domain this domain can be malicious or can be benign we want to decide it we want to construct a classifier that would predict that um with regard to graph neural networks is it correct then to say that the scalability of message passing is one of the main limiting things where that technique can't tackle the problem yes yes what gnns do that is that they apply several steps of message passing and that is really really expensive for this this type of we had graphs or we had domains represented as a ver vertices in these graphs that had hundreds of thousands of neighbors you can't you can't compute that with with GNN so we we took a different approach here so you have this very interesting topology is a graph but it's not obvious what's malicious and what's not how do you get some labels here yes that's that was a different different challenge that we also had to solve when writing the paper and that is and this is a challenge in all network security is that you usually get some kind of deny list which is uh curated by your your analysts and you want to extrapolate from this deny list so what we know when training is that we have this graph and we know that some of these some of these vertices are malicious domains and we want to take these as labels it's a problem that in machine learning is called positive unlabeled problem because we know some vertices in the graph are malicious are positive but we know nothing about the rest so this the rest is highly likely benign because if you took if you take a random random vertex from the graph or from the internet random second level domain it's likely benign because because of the prior but we don't know that for sure so that's one one problem another problem is for example that you really have to be careful when evaluating your your algorithm so let's say that we have a domain abc.com and you have the.com and you you know that the first one is malicious the other one is not and you train your model on a week of data for example and then you want to test your model on on the next week so you you collect the data and you predict the label for each of the each of the domain represented as a vert in this graph but then you need to be really careful if these if these two domains that you had in your deny list if these appear in in the next week of your data because you must be careful to learn the specifics of malicious behavior and not the specifics of this of this domain instead if there is something really specific in the abc.com domain if there's something really specific you can you can learn that your model can learn that but it's not really useful you know because other these malicious domains they pop up and disappear very quickly so you really need to do to learn the behavior and we also employed the technique to test that the that the model is immune to this problem and then the model's output uh what does it give you is it's you is it labeling the new vertices just uh binary classifier is there a probabilistic scor yes it was it was it was it was binary classifier with some confidence yeah you can you could and then uh I'm picturing the graph of anyone who's on the deny list is known to be you know a malicious node and then somewhere between zero and 100% of the other nodes are going to get labeled malicious what is the output uh could you give us just some sort of summary analysis on uh the frequency with which you discover it or something along those lines how much ious data is there to be discovered it's it's usually less than less than a percent of all of all the incoming domains this is one possible application of of this of this model or of this technique is that you can use it to generate uh generate possible malicious domain candidates so you can run your model and take the top say 500 domains daily and send it to your analysts and you can you can largely reduce the chunk of work they have to do they can just focus on these very suspicious domains well in a general machine learning case it's very nice when you can look at your uh holdout data set and say you have type one errors and type two errors and F1 score and these sorts of things but you have a somewhat unique challenge in that there isn't a perfect ground truth how do you look at performance metrics there are some specifics that apply to whole cyers SEC field when of this is that ground truth is is expensive I wouldn't say that it's it's not of a high quality there is Grand truth which is available but it's sparse and scares so you need uh for example when classifying images everybody can tell whether an whether an image is a dog or a cat but to classify a domain you need to have professionals experts with their tools it's sometimes very hard to tell especially if you're not like core cyber analyst and you're you're you're doing machine learning it sometimes you need to cooperate with others so this is something which uh which makes this domain more challenging than I would say images for example because when you're developing a classifier for images you can always check whether it makes sense you can you you you know you know what what the image shows I wouldn't say that you need to that in cyers sec we use different uh set of metrics it's pretty much the standard and the standard standard set and in this case when where the prior of malicious domain is is slow so there are much much much more benign domains than not you can look at it as some document retrieval so precision and recall are perfectly valid metrics here what I would say is that you can't in in cyber SEC uh false positives are really expensive mistakes to make what we usually do is that uh we plot r r curves and we look at areas with low false positive rates low false positive rates usually plot uh plot RC curve with logarithmic logarithmically scaled X x-axis and you are interested in this in this area but these are also quite standard standard metrics yeah I would say yeah do you think it has a path to Industry is this something that will improve the quality of cyber security yes definitely definitely uh this was uh as I said this was developed with Cisco so this was tested on real world uh production data so I think the the potential here in this regard is huge could you outline maybe uh for listeners sake some of the wins that'll come there I would imagine there's accuracy gains from new techniques but I think there's maybe even some benefits in um being able to quickly create a new embedding rather than hand coding features what do you see as the major wins for uh getting this into the cyber security World there is a stark improvement with respect to the previous state-of-the-art algorithm which was used for this kind of problem which is this this algorithm is called probabilistic threat propagation and even though it works quite well for for some problems it has some serious limitations so for example what probabilistic thre propagation can't do is to employ multiple binary relations I spoke about clients and domains then about binaries and domains you can observe multi multiple such binary relations and work with them probabilistic threat propagation can only process one of these binary relations s simultaneously we employed more of these more of these relations so we took if I remember correctly like 11 of them so this was clients these binaries we also had some TLS certificates all of this we we put this into the framework and this dramatically in increased the efficacy of the model it was like threefold I think so this was a very very good Improvement in efficacy I would say and also it was really really nice that we were able to apply neural Nets and these uh these modern techniques from machine learning to to this kind of a problem the hierarchal multi-instance learning then had a good success in this domain are there any other areas you look forward to maybe applying it to in the future or perhaps other projects where you currently are the our our plans currently and the pl of our whole group is to aim to for cyber security So currently for example we are preparing a paper for usik which is not which is rather cyers SEC Conference rather than core machine learning conference cyers SEC is a really really fitting domain here so we would like to first get get feedback for our work obviously and also to show show our work to the world with regard to Jason grinder that we talked about earlier could you you share some details on it as a tool is it something you think becoming a project that other people could use maybe something open source or what's the state of it we have two main libraries which are open sourced completely they are written in Julia Julia language which is very similar to python so anybody can try them we have two two components one is one is Jason grinder which as the title suggests can process these Json the Json grinder can infer schema and prepare the Json into into the structures that I described so that the model can process them and then we have something which is called mjl multiple instance learning library JL which is another library that takes care of this Pro processing so you can build a model there and process these uh these data these are two core libraries that we have we are preparing the third one which is about explainability so nowadays it's really important to be able to explain the results of your model and we have some tools in machine learning but they have some some some big problems for example that they do not really Faithfully describe what the model actually computes this is still an open problem I would say in machine learning and not fully solved at all and it turns out that the way we process the data essentially skipping the vectorization of the the data the feature Vector mapping it's a really great Advantage for later explainability because you have the whole whole input data at your disposal and you can you can create explanation out of that so this is the first the third Library which is it is open source but uh it's it's in development currently it's called explain Mill and it's it will be for explanations we have extensive documentation with a lot of examples so anyone who has some Json documents at hand can try it shiman what's next for you yeah I want to I want to publish more in in the cyers sex space and to promote our work and obviously finish finish finish the PHD and then we'll see very cool and is there anywhere listeners can follow you online I have Linked In profile and I'm also believe on Google Google Scholar so I'm there yet other than that I don't use uh I don't use um social networks that much Google Scholar is always a good spot thank you so much for taking the time to come on and share your work yeah thanks for the invitation it was a really nice talk [Music] [Laughter] [Music] [Laughter] [Music]