Master Data Management Fundamentals | Data Science | Community Webinar
Key Takeaways
This video covers Master Data Management fundamentals for data science applications
Full Transcript
So, what a data scientist need to know about MDM. I'm going to move pretty quickly through some content dense uh slides here and and then I'll take the slides off and we'll dive into Q&A or please ask questions as we go. Let's start really quickly with the definition. This is Gartner's definition. So, I come by this definition and all other Gartner insights. Honestly, I worked for Gartner for three years. I was an MDM analyst. I was a data governance analyst with Gartner. So, I participated in creating this definition. Um it's really if you if you look at this it's really all about a couple of things. One is governance right being able to apply consistent policies and procedures to ensure the uniformity accuracy consistency semantic consistency of an enterprises shared data assets. The shared data is really really important. If data wasn't shared we wouldn't need MDM at all. First and foremost MDM is a discipline. It's a way of managing data that it's a technology the technology MDM software supports the discipline you can you can do data governance without MDM um but you really can't do MDM without data governance MDM is an implementation and execution layer inherently governance of the rules governance of the policies and procedures MDM as a discipline and as a technology is used to implement those rules a and again this is really all about shared data you know the shared data kind of objects or domains that are are widely used across the organization. People, customers, employees, locations, products, you name it. Data that travels from function to function to function across the entire organization is really the realm of master data. It's having common policies and procedures that that that that uh rule that govern this data. I hate to use rule because it sound like control. Um, what problems does MDM solve? Well, really what what what data scientists should know here is that that MDM solves both analytical problems, right? It helps makes analytics better. It's going to help make your models better. It's going to help everything that you do be better from an analytical perspective, but it's also used to solve operational problems. So the the the kind of the quintessential problem that MDM solves is the single version of the truth. MDM as a technology is a data hub, right? the the old school architectures for MDM are the hub and spoke where you create and persist a master record in some hub and then you then you distribute that hub either to analytics platforms or to operational systems. We'll talk about that a little bit more. As companies grow, it's just kind of natural for data silos to evolve over time, application by application, function by function. The way that sales and marketing defines a customer is different than the way that finance defines a customer. And those rules and quality standards will will will differ just kind of naturally. It just happens or companies will go buy other companies and and these silos just start to pop up. Yeah, you can pull all of that data out of the source systems and drop it into a lakehouse, boat house, swamp, whatever, but it's still not going to solve putting it in a single place is not going to solve for having different definitions or different labels or different addresses or on and on and on. That's the problem that MDM is custom built to solve. Right? inconsistent, inaccurate, unfit for p purpose, incomplete, lowquality, varying by source. Right? So keep in mind here that MDM is about analytical applications and it's also about operational applications of this data. Right? MDM can actually be used as a technology and as a discipline. It can be used to solve for the garbage in problem. We'll talk about the kind of the two forms of MDM here very quickly, but it can be used. You can deeply integrate MDM as a data hub into a source application like a CRM or ERP or wherever data is created created or updated to solve for the garbage in problem. There are really kind of two ways to approach MDM. One is what's called an analytical style of master data management where you are applying consistent data governance policies to a given data domain or a data object where really what you were solving for is creating a single single instantiation of that record where you could actually create some form of a new master record. But really all you're trying to do here is to link like records together. Acme Acme Incorporated Acme Co. Acme and suns is that one thing or is that four things? MDM would define the rules that allow you to confidently answer that question. Is it one thing or is it four things? Right? What is our definition of uniqueness? What are our quality standards for accuracy related to that data? Those rules would be applied by MDM and allow you in an analytical style of MDM to at the very least link all of those records together under some new master ID that references all of those source IDs. So that in a downstream system like an analytical platform that you guys may be using day in and day out, you can refer to that master id and you could even aggregate correctly by because all of the source ids would be associated to that master ID. Often in an analytical style of MDM, there is a master record. Some new master record is created, but often it's just used kind of as what I call a stub that simply it's a reference. It's a key back to those source IDs. analytical styles of MDM MDM solve for the 360 of something, right? Where you could correctly and consistently and accurately aggregate data for suppliers or customers or locations or employees over and over and over again. Technically, any database could be used as as a kind of an MDM persistence layer here as long as the rules that you were applying were consistent over time. But generally what most large companies are doing is using MDM software to deploy these types of analytical styles of MDM. The pro here is that these styles of MDM analytical INDM can be done pretty quickly. Really all of the kind of the heavy lifting here is is entity resolution entity resolution like the algorithms that are applied by MDM software to understand if Acme Co, Acme Inc., Acme and Sun is one thing or three things, right? you can generally run these processes pretty quickly. You're not changing any source data, right? You are not applying really kind of complex rules around merging data. All you're trying to do is to link things accurately together. So these things can be deployed pretty quickly. Um the con here is it's not going to fix any data, right? If you've got data that is is badly duplicated by source, implementing an analytical style of MDM isn't going to solve for that. It's not going to stop the flow of garbage in. still trying to kind of catch things after the fact. But at the very least, you can deploy some analytics using these insights. You can deploy analytics that that that put a single view of customer or supplier or employee or location and on and on in front of end users. Operational MDM is is the opposite of all this, right? It it is where you would create some new master record that is an amalgam generally of all of those source records and then you take that record and you syndicate it back down into operational systems like CRM like ERPs like other systems where data is managed day in and day out and you can push that record back into some sort of analytical platform that you guys would use to build your models. So really this is a harder style of MDF because it can tend to be very disruptive and often even destructive meaning where you are merging records together. Now I would never recommend that you destroy the source records because you always want to be able to roll back. But when you take two things and make one thing, right, and then you pump that one thing back into an operational system, end users are going to see something different than they saw yesterday, right? Right? If you create a new record, this is the new master record that is used within an ERP system or a CRM system. To build those rules is pretty complex, right? We call this survivorship rules. These are the rules when matching things together. So that's one set of rules. Then the second set of rules is what do we use to create this new master record at a field level or a record level? Which source system wins, right? How do you how do you promote field by field by field into that master record? Developing those rules can be a fairly complex process. Ultimately, what we're talking about here, guys, is some form of a data governance committee to decide all of those rules, but companies can wrestle with that and and and it can be a fairly disruptive process, and it can take a long time. This is kind of classic MDM. Most people when they hear MDM, oh, single version of the truth. We're merging records. We're picking rules that, you know, for survivorship and on and on. Kind of older school to the point here at the bottom of this slide. kind of older school MDM, but there are still some companies that that are following these paradigms, right? Particularly manufacturing companies, other companies that are making things that need a lot of control o over data, right? Um, so this is still being done widely. By the way, this is basically the model for reference data, right? Reference data, single source of truth. Everybody uses it. You defer to the the single source of truth. Um so you know weights, measures, dimensions, those types of things uh kind of are operational forms of of of MDM that are tightly controlled and managed. So kind of the pro here, the good side here is is that you're fixing the bad data. Uh you are technic generally fixing it after the fact, but you are resolving for the bad data. Something that was bad or incorrect or low quality is now higher quality that you're seeing it in an operational system, right? Not just an analytical process or an analytical platform, but in an operational system. The downside is it can be disruptive. It can be timeconuming. It requires higher levels of maturity when it comes to data governance because you're going to need to define all of these rules. Um and it and again it can be seen kind of old school. Um we'll talk about that a little bit more. Uh high level this is really really high level architecture. You can you can tell I'm not a systems architect. Left hand side you know data is coming out of sources. It's going into some MDM hub but the data doesn't go back into the sources. a one-way flow of data into a data hub and then analytics platforms whether that is a lakehouse, boat house, swamp, whatever or any sort of platform that you may be using from a data science perspective generally would be consuming that data, right? would be consuming data on those kind of those master ids and maybe even the master records where you could use those IDs to key key back into source systems or key back into any sort of other kind of you know data warehouser or or or similar to do correct aggregations related to employees customers locations you name it other master data domains so I'll take a breath in a little bit of water any questions so far I'm looking at the chat I don't see any I have not seen any questions come in so far. So, um, typically Okay. Yeah. typically what happens is we stop for questions and then you'll start talking again and then we'll get questions. Okay. Well, because I'm throwing a lot out there now. I'm kind of following the the shock and awe approach here. He's throwing a lot of content out. But I I do want to cover it all and then we'll we'll take the content off the screen and be a little more organic. But so those kind of two highle approaches organic operational MDM and analytical MDM. Um, Gartner says that there's these things called implementation styles. This this kind of gets into what do you persist and what is the system of record. In an analytical system of MDM, the system of record generally is the source system still because you're not really kind of changing anything in an MDM hub. In an operational style, the system of record generally is that MDM hub for for certain domains. But these are the four styles. consolidation style that really aligns to the analytical MDM that we were talking about where all you're trying to do is link things together. Centralized styles of MDM. This is old school hub and spoke, right? Where you create a master record and persist it, manage it in in some sort of hub and the hub then feeds all the downstream consuming systems. A registry style is MDM is is a really rare form of of MDM and I glibly said here in the notes here that that hasn't been seen in the wild in years. I worked for at Gartner for nearly three years and I talked to 1500 companies never saw a single registry in the wild. they tend to be fairly complex where you're using that MDM hub basically kind of as as as just as it says a registry where you maint acts basically that MDM hub acts as a really really kind of sophisticated integration layer where it kind of in real time where systems can be deferring to other systems as the source of record and the MDM is saying hey go over here for for for the master record for A or go over here for the master record for B it it's it would be used in highly virtualized forms terms of data management. Haven't seen this in a long time, but I think we could see this kind of kind of revitalize over the next few years as we push more towards data fabric type type architectures that are highly highly virtualized where some sort of advanced semantic layer is controlling MDM is and is controlling a lot of these governance policies and and we'll talk about data fabric maybe a little bit um because today MDM is still you know going to be required for data fabric or data mesh Um, style number four, obviously some sort of hybrid of the of the top three. Um, what makes MDM software kind of unique, right? What what what do you buy? If you buy MDM software, what are you getting? Well, using a Gartner lens here. Gartner defines enterprise software solutions through these things called critical capabilities. 10 of them are listed here. Um, I know these very well because I wrote the last MDM critical capabil capabilities document for Gartner. I won't go into detail here, but to make a long story short, to be considered MDM software, you got to do all of these things. You got to be able to support multiple domains. You got to be able to support multiple implementation styles. You've got to have some data quality capabilities, some workflow capabilities, data governance, on and on and on. If you're not checking all these boxes, you then you're not MDM software. You're you're something else, right? So, I see a lot of platforms out there that say, whoa, we can enable a single version of the truth. I mean even even data warehouse solutions um even like the palunteers of the world are out there saying well we can support MDM well probably not right so there's a lot of solutions out there that are saying well we can do MDM unless you can check all of those boxes you're probably not doing enterprise class MDM in the case of number one here from an analytics platform perspective just adding in like permissions and access control uh is not MDM and putting data into one place and adding some layers over top of it for permissions and access is not MDM. MDM is all the other things that I showed that I shared on the last screen. Yeah, it's a good start to put all of your data into one bucket, right? Yeah, it's a good start to have, you know, controls over who can see it from an access perspective, but that's not MDM, particularly operational MDM, right? So keep in mind MDM is inherently about deep integrations to operational systems that are creating and editing data. And if you're dealing with an MD, if you're dealing with a kind of a, you know, lakehouse, brick house, swamp house, whatever provider, and they're saying, "Oh, well, yeah, we could do MDM." They don't. Uh, nor do other point solutions such as data customer data platforms. Customer data platforms kind of get close. um they've got some light MDM capabilities but they fall short in a number of key areas particularly the governance particularly complex hierarchy management I haven't talked about that but hierarchy management is also something that is generally managed and and maintained and persisted in an MDM hub uh complex relationships that exist between entities particularly things like um customer hierarchies uh in particularly in the B2B space like the hierarchies that exist within corporate structures like parent child type relationships So, so hierarchy management is a key capability of of MDM customer data platforms do that very well. CRM systems um you know Salesforce is out there saying hey we we we support single version of the truth we support a 360 view that's true for a single use case for a sales and marketing use case but again they're not going to be checking all the other boxes that were on the previous screen here. So Salesforce is really good at doing CRM and they're really good at providing some form of a of a 360 degree view or customer data. So one domain customer data one use case sales and marketing right that is not what an MDM MDM is about creating those single views for enterprisewide use cases for all use cases for all data domains right so you could create a product master you could create an employee location and on and on that's what MDM does across multiple use cases not just sales and marketing but finance accounting compliance procurement supply chain manufacturing You name it, MDM software is built to support all of those use cases and all of those data domains. So Salesforce can enable a 360 of something. Yes, it can. But it's a customer- ccentric view supporting one specific use case. Uh the number four on here, there are data governance platforms out there and data quality platforms. A lot of the data governance providers um particularly those that are well all of them focus on building and managing data cataloges. Building a data catalog is not MDM. It's just not right. It's a good start. And don't get me wrong, data cataloging is really really important, but it's it's not a single source of truth. It is not the ability to configure and manage complex data quality rules, complex matching rules, complex governance rules. You're not well, yes, governance, but typically MDM is actually implementing and executing the governance policies, many of which would be housed in some sort of a governance platform. So again, MBM is really all about operationalizing a lot of governance policies, particularly those relevant to master data, shared data across an enterprise. So not everything that can do some MDM capabilities. That's part of the problem here is there's a lot of overlap. We were looking at a vend diagram, you know, data quality solutions, data governance solutions, data integration solutions. there'd be a little overlap into MDM, but can they enable all of those other capabilities that I was talking about on this slide? Uh, generally they cannot. Um, man, particularly younger folks in the data and analytics realm, which a lot of data scientists are, and that's awesome. I hear this all the time, man. It's like, oh, MDM, that's that's old school, right? I MDM's dead, isn't it? Isn't it going away? No, it's not going away. I would argue as long as there is a need for enterprises to share data widely at scale within operational systems there will be a need for some form of MDM right there's a lot of talk out there now around data mesh and data fabric I would argue if you are going to implement either of those architectural patterns and yes I can hear the data mesh acolyte saying but it's more than just architect ure it's a sociote techchnical okay fine if you're going to implement that you're going to need some form of MDM data mesh is an analytical approach it's not an operational approach but even in the analytical realm you can be very data domain centric you can give sales and marketing complete control over the data we did that 20 years ago by building these things called data marts separate issue um but you can give control to a domain over data but the minute you have to share those analytics across functions. The minute your CEO asks, "How many customers do we have?" There can only be one answer. And trust me, you don't want to be the one that goes into the CEO's office and says, "Well, who's asking in what context? For what use case? What do you mean?" Not the right answer you want to give to a CEO. So, as long as we are dealing organizationally, crossf functionally, which we all do, then you need to have a single answer to that question. I would argue both the mesh and the fabric have not come up with a compelling technical solution to allow for crossf functional views and governance and management of data in the mesh. I I there's a notion of this thing called federated computational governance but I don't know in practice what that really means and and I don't mean that to be mean. I just I just don't from the perspective of a data fabric which is different than a data mesh. Separate conversation I'm happy to have. There's this kind of kind of handwavy semantic layer plus maybe a little bit of governance plus maybe a little bit of scale and AI and ML plus automagic integration that happens because of active metadata equals what? Well, that I would argue what we just said is MDM, right? And I haven't seen yet a single company. I've talked to hundreds and I haven't seen yet a single company that has that has built some sort of technical layer that would allow for in a data fabric architecture allow for widespread operational sharing of data across data domains that are widely shared like customer or product or location. So I would argue that if you want to do either a fabric or a mesh, you need MDM and focusing on fabric or mesh does not mean MDM goes away. MDM as a discipline, the ability to share data and have consistent definitions, consistent structures, taxonomies, hierarchies, all of those things to do that in a consistent way that will give your sea level uh executives confidence in the reports they're looking at. Uh you're going to need MDM for that. Digital transformation, um what we're seeing, MDM is not dead because digital transformation efforts are increasing and are accelerating and there's more and more and more and more data than there ever has been. And there's more data today than there was yesterday. All these things are correct. And what we're seeing is companies are rightfully focusing on digital transformation and they're saying, "Okay, we want to change how we interact with our customers. Who are our customers? We we want to change how we we procure the goods for our products and how we build our products. And we want to understand what people are clicking on and not clicking on across channels and and and and across products and across divisions and all of that. You need a foundation of MDM to do it. I would argue if you don't have a single consistent trustworthy source of information about your products or your customers or your suppliers, you're going to be hardressed to get any correct analytics out of a system that doesn't allow for consistent management of the governance policies over those data domains. Yeah, AI, ML, graph, the stuff that you guys are probably dealing with every day uh is is leading to a lot of really cool stuff and we are seeing more automation in this space than we ever have before. We can build automation around building master data models. We can automate some of the data stewardship capabilities required here. We can understand complex relationships like we've never understood before. Graph in graph plus MDM equals goodness, right? Because we can understand relationships that we never even kind of pointed our our data models at in the past, right? In the past, MDM was very top down and we could say here's how we define customer and here's the data model needed to for that and here's the the entity resolution needed for that and now with graph we can actually take more of a bottoms up approach where we can go and understand well hey we weren't even looking over here before but it's relevant from an MDM perspective or a master data perspective so there's a lot of goodness there but it's not replacing MDM because you still need to consistently apply all these business rules as I mentioned MDM is changing let me do a time check here 123 Oh, that's good. Can have a decent amount of time for uh decent amount of time for questions. And I do see that Zan has one. Um well, okay, we're between slides. We can we can answer it. Does any automated automated data sanitization? That sounds like a dirty job. Data sanitization. Um yes, it does Aan. And it can support uh for a limited set of data quality rules for a limited subset of data. So, MDM is not meant to be an enterprisewide data quality solution. It is meant to allow for certain aspects of data quality to be maintained for a subset of data, right? Master data, customers, employees, locations, products. When you define what data is master data within your organization and really what that means is what data is widely shared that needs consistent quality standards that needs consistent structure taxonomy governance rules applied to it. You can use MDM to enforce those rules. You can say things like here's how I define uniqueness. Here are the fields that are required on this. Here are the field types that are required within each of the the master data records or objects. So and and you can use MDM to do data transformations and to do to to apply those rules consistently consistently across uh data sources but again it's for a subset of data it's for it's for master data and it's not meant to be kind of a data quality tool for all fields all objects all use cases that's really the realm of a data quality solution so MDM solutions and data quality solutions will work hand inand where data quality solution will generally defer to MDM and for the management uh of the business rules related to customer product location you you name it. So good question um MDM's not dead not going away but it is changing. um we no longer really kind of say single version of the truth cuz cuz number bullet number two here because context is increasingly king and you guys probably already know this but I mean like context is way way I mean it's critical right in the past old school MDM when most people think of MDM 20 years ago there was one context it was the enterprisewide kind of topdown seale view of the world. That was the context that matters. That that mattered and and that's kind of how MDM systems were built in the beginning. But now what we're seeing is that context is increasingly important. Sales and marketing can look at those master data domains or objects differently than everybody else can or finance or accounting or compliance and and all of those views are correct at that functional level. Right? So th those views are correct. However, when you move up the organization from functional to cross functional to enterprisewide, those definitions, those quality standards, those views can change naturally as they should, right? And MDM is evolving to be able to support all of those use cases, right? All of those kind of views of data, all of those uses of data. So that what you see within an operational system at a functional level could be different than what you're actually seeing in an operational system at more of a cross functional or enterprisewide level. So we're moving away from this idea of single version of the truth towards more of a single source of truth. MDM still largely is a data hub architecture, right? It is a single single place to get these insights from because there are economies of scale from managing these business rules in a single place. Right? There are still economies of scale that exist from hub and spoke type models versus spiderweb models. This is another thing with the data mesh I just I just have a hard time getting my head around which it just seems like outside of a domain there's a lot of spiderw webs to be built. Separate issue. um context is king and we're getting away from the idea of a single version of the truth into at least a single source of the truth but where you can have kind of different instantiations. You can have different views. You can have different uses, different access paradigms and on and on. There are more kind of adaptive models that are evolving in the MBM space um because of more of a focus on what we would kind of kind of roughly call adaptive governance which really just acknowledges that context matters. In the past, governance was one-sizefits-all thou shalt type approach to managing rules on data. As MDM is evolving, so is governance. Governance is evolving to be more contextual and have multiple sets of rules. Now, the challenge there for a lot of organizations. It was hard enough to have one set of rules for a lot of these companies. Now, you're talking about n sets of rules. Uh harder to do. Um, you could argue that those rules have always existed at a functional level, right? Sales is going to do what sales wants to do, marketing is going to do what marketing wants to do and on and on. But kind of coraling all of those at least under a common governance model and understanding what they are, even co codifying or even just documenting some of them for a lot of companies is just a challenge. Uh, and yes, AI, ML, by the way, I know these are two separate things, artificial intelligence and machine learning. Uh, and some some of the purists kind of get get upset when you lump them together, but understand their different things, but they're really kind of pushing the boundaries of of what was historically considered MDM. And the way that I look at it is before old school was top down, right? It was here's the data model, right? Here's our customer data model. Here's our product data model. And here's all of the rules that we need to kind of manage to that data model and all the policies we need to, you know, make sure the data is consistent, accurate, trustworthy, and on and on. What's happening now is with these new technologies that we can take more of a bottoms up approach because these technologies allow for scale and they allow for automation. So this is a really good thing. You can have the top top down approach because again if the CEO asks how many customers do we have there can be only one answer. However, but working from the bottom up as well you can you can learn so much more stuff now than you can ever you ever could. The example I always give is you know running writing a knowledge graph to understand relationships that are meaningful that you didn't know about before. An example, customer data historically when taking that top down approach was here's the customer, here's how we define the customer, right? Maybe it is a person plus an address or a company name plus an address. Um, running a knowledge graph, you can see, oh, wait a minute. Hey, there's a there's a relationship between these two customers that may be a family relationship that you didn't even know about before. Um, that's interesting. And you can actually start managing that relationship as a master relationship. So you can start applying kind of more consistent business rules and governance policies to that relationship when in the past you wouldn't even considered it because you didn't even know about it, right? You just kind of had this monolithic top down approach to what a customer was. Now you see something else. Oh, this is cool. let's let's apply some rigor here and start managing this more as a master data relationship which is uh which which so it's the scale it's the insights it's the automation that these new technologies are offering that is really kind of pushing the boundaries of what was considered MDM or what has historically been considered MDM what does the future look like well we're going to see more automation we're going to see more particularly around kind of data modeling in the MDM space we're going to see more automation around data management data stewardship ship auto detection of what is and what isn't master data. Um, you may have got the impression that I was a, you know, a bit of a naysayer around the data fabric. I'm not. I I actually think that there's there's some something there. I love the idea of active metadata. I I love the idea of data itself informing its own classification and its own use, meaning governance policies where data can tell you what the appropriate data itself can tell you what the appropriate governance policy is. That's going to enabled by active meta metadata. So you could get it to a place very soon here in the future where the data tells you what's widely shared and what's not widely shared. where you could query large troves of transactional data to understand where is data being widely shared across the organization that's already being done today. So instead of a a person say you know here's what's important to us. Um you could have the machine say this is what's important to us. Maybe you guys are working on something that looks like that. It's pretty cool automated development of hierarchies and relationships. We're already starting to see some of this. I think you could start to see kind of more virtualized master data hubs where instead of it being a physical data hub that that that could be kind of broken and virtualized where you could have kind of mastery of data what I'm calling in situ where where the data lives right and with more flexible data models that's something that we're actively working on at prophecy um some exciting stuff there where we start to get away from you know collecting data and instead to more connecting to data um which which I think is is kind of cool. I think we could see that in the very very near future. Uh deeper integration uh across other elements of the the the data estate today data management solutions, data quality, data integration, data governance, they can they kind of exist to a certain degree, not entirely uh in silos. Particularly data governance platforms like the elations and calbras of the world or the Informatica Axons of the world can tend to to live in a little bit of a silo. We're starting to see deeper integrations between solutions. Uh an example in my world is is MDM like Prophecy deeply integrated to a data catalog data governance solution like a Purview by Microsoft. So we're starting to see these things start to come together where you could have policy definition and policy management and policy execution all happen kind of seamlessly across two solutions. That's pretty slick. And yeah, blockchain. Um, I'm bullish about blockchain in the future of data management, particularly MDM, because I think when you look at blockchain, what we're really kind of talking about here, blockchain, if you ask me, is a data architecture, no more, no less, that has an incentives layer on top of it, but it's basically a data management architecture. You you've got ledger databases that are really good at a few things, including the management of governance policies, right? where where where the participants in that chain can actually vote on governance policies. What do we accept? What do we deny? And actually be paid for it, where you could even be paid uh incentivized to do data stewardship on a shared data ecosystem because that's what blockchains are. These it's a peer-to-peer data network in essence. If you disagree, we'd love to hear different uh uh perspectives there. But I could foresee in the very very near future where certain use cases in the data management realm particularly kind of the heavily you know focused on lineage. Um where where where certain forms of reference data that are heavily reliant on controls over lineage and other kind of data management paradigms like that kind of evolve using blockchain. Some some examples here could be like land titles right uh filing and managing like a title or updating a title on a land or or leans against land. These are some use cases. The use here is is long, but I do actually see blockchain kind of coming to MDM sometime in the near future. I'll end on this slide. Give us a few minutes to talk. Um, does some of this sound familiar? I I suspect that there are data scientists out there that are doing MDM or or some form of MDM without knowing it and even without knowing that they could push back to other areas in the data analy data and analytics world to say, hey, maybe we should be looking at off-the-shelf software here because I'm spending a lot of time focused on data quality problems. I'm spending a lot of time building. I've even built some custom programs or some custom software or scripts to do complex entity resolution. I'd be willing to bet you have, right, where you're kind of doing solving for the Acme Acme Acme Co. Um, when maybe your time is better spent doing something else because there's an off-the-shelf solution, offtheshelf solution that can do this stuff pretty well. It's MDM, right? I bet you're spending a lot of time building, implementing rules to transform data. We already know this. Data scientists spend an inordinate amount of time transforming and normalizing data. Right? Again, there's there's a solution for that at least for some domains that is in some data that's widely shared, MDM. So, if you're spending a lot of time doing these bullets, data quality, transformation, normalization, I suspect that's what led to that first question. uh or if you are like working and spending a lot of time trying to figure out identity resolution or entity resolution, maybe you shouldn't be because there's software that out there that does this stuff. So, if anything that I've shared with you today makes you kind of question maybe I shouldn't be doing that. go have some conversations internally with with folks that are, you know, kind of leading the data and analytics function at your organization because maybe some of the stuff that you're doing that is lower value ad could be kind of pushed off uh towards an off-the-shelf solution like MDM software. So I'll end with that. Okay, perfect. And we do have a question. Um Karen, so what is the difference between single version of truth and single source of truth? Um well a single version of the truth is is one customer master record or location employee doesn't matter but one record that is generally persisted in some sort of data hub where that one record then is syndicated everywhere operational systems and analytical systems where the control of a new customer record or a new product record or a new location record is tightly tightly tightly controlled and tightly monitored and only certain people can create new records. records and edit those records. Classic hub and spoke single version of the truth and it's it's consumed by everything. Every analytical platform would consume that single version of the truth and every operational platform would consume consume that single version of the truth. Uh a single source of the truth is we're just getting away from single version of the truth because of the things that I mentioned earlier about context. Right? Sales and marketing don't want to be told how to run their business, right? and they don't want to be told you're going to use this version of a customer record whether it fits your needs or not. Right? Typically those kind of single version of the truth approaches take kind of down and they will take the kind of the lowest common denominator definition of quality, definition of uniqueness. generally the the definitions that that best align to how seuite executives want to see things and they'll say okay here's how we define a customer right in the B2B realm a common approach here is to use a third-party data provider like I've done in Brad Street to say here's how we define customers we're going to defer to D&B this is our customer definition now everybody has to use it that's the single version of the truth everybody has to use this whether you like it or not what we'll see is the companies like hey certain division or functions or will say no this doesn't work for me it doesn't align to how I sell it doesn't align to the how I manage my customer relationships I want a different version I want a different definition I want a different structure to my customer def to my data so single version of the truth that's it top down one version of one of these kind of domains or records to rule them all single source is just a place to go to get insights on maybe it's a single version but generally increasingly what we're seeing in the MDM world is is multiple versions of the truth, but where you at least go to one place to get those insights where the MDM hub is smart enough to know, oh, okay, this is a sales and marketing use case. Well, then I will serve up this instantiation, this version, this view of of a customer record. Oh, this is a different use case. Well, then use this version of a customer record. So, there's still kind of an architectural advantage here to going to a single place to get those insights instead of having to go spiderweb and try to find them within the organization. That's really kind of what I meant by kind of a single version versus a single source source. Hopefully that makes sense. Okay. And I have I also have a question for you. So when you were going over the MDM styles and and maybe you may have gone over this and I was just busy doing something else, but when do I know like how do I know what the best style is for my organization? There's that's a great question, Nathan. Um there there's a few different answers to that. One starts with of course the requirements, right? What problem are you trying to solve? Um are you trying to solve for only an analytical approach? And sometimes that's valid. I'll give you an example. Bergkshire Hathaway, massive company, but it's a holding company, right? Bergkshire Hathway corporate doesn't really have any operational control or at least they choose not to have any operational control over their operating divisions. The operating divisions have complete and total freedom to do whatever they want. Right? Berkshire Hathaway implementing MDM. They're going to be implementing an analytical MDM because all they're trying to solve for is is consistent insights across their operating units. They're not going to be enforcing the operational consumption of master data. Right? So it starts with requirements, but it does get into things like your overall operating model, right? How decentralized are you from an operations perspective versus how centralized are you from an operations perspective? Manufacturing companies tend to be fairly centralized because that's kind of how they're born and bred, right? Where they are very process ccentric. They're very control ccentric. Sometimes they may even be making things like medical devices or jet engines or other things that require an incredible amount of precision and control. Those operating environments, those business models kind of lend themselves naturally to more kind of top- down command and control approaches to data. There are other companies out there that may be more technology ccentric that may be more consumer package goodcentric or or or likewise where they are highly autonomous where at a functional unit functional units have a lot of autonomy or control over what they do and the data that they manage. So your business operating kind of model will play into this as will your overall data strategy. Generally what we see however is that there isn't any one style of MDM implemented. There are multiple right for some use cases. You want control, you want top down, you want a lot of oversight. This is the generally the realm for more finance or audit or compliance or regulatory driven use cases where you want a lot of control. You don't want anybody willy-nilly just editing records wherever they can. In those environments, that would call for one style of MDM. Then there are other use cases like sales and marketing use cases where the cost of being wrong, right? It is generally pretty low, right? Where it's more about scale and it's more about agility and it's more about business freedom where okay, you've got two records for Joe Smith. Well, what's the worst that can happen? Well, we send the wrong offer or our marketing programs are inefficient. It's not optimal, but you're not going to get sued. You're not going to break the law. You're just going to be a little more inefficient. But you may be willing to trade that off for the freedom and the autonomy to operate however you want to operate. So there's no one kind of one answer to that question. But it really depends on your the overall requirements, depends on your operating model, depends on your data strategy, and it depends in many ways on the data itself. Reference data just naturally aligns to kind of centralized controls and centralized oversight of data. But then there's other forms of data not widely shared or only shared on a limited basis that could be managed a little more loosely, shall we say. So again, Malcolm, thank you so much for being here. Thank you everyone for joining us this morning and I hope everybody has a good rest of their day.
Original Description
An overview of Master Data Management (MDM).
A lack of Master Data Management (MDM) can spell disaster for the accuracy and predictability of any models or analytics built on poorly mastered data, yet many in the data science world are unaware of the role MDM plays in their daily jobs. This presentation provides an overview of the discipline of MDM and the technologies supporting it and provides some recommendations on how data scientists can leverage MDM tactics to drive improved business outcomes.
0:00 Intro
0:12 What is MDM
1:48 What Problems Does MDM Solve?
3:51 Analytical MDM
6:52 Operational MDM
9:46 Overview of Both Approaches
10:33 Implementation Style
13:13 What Makes MDM Unique
14:16 Solutions to Support MDM
18:24 Need for MDM
23:38 MDM is Changing
31:04 Future of MDM
37:01 Q&A
For more captivating community talks featuring renowned speakers, check out this playlist: https://youtube.com/playlist?list=PL8eNk_zTBST-EBv2LDSW9Wx_V4Gy5OPFT
--
At Data Science Dojo, we believe data science is for everyone. Our data science trainings have been attended by more than 10,000 employees from over 2,500 companies globally, including many leaders in tech like Microsoft, Google, and Facebook. For more information please visit: https://hubs.la/Q01Z-13k0
💼 Learn to build LLM-powered apps in just 40 hours with our Large Language Models bootcamp: https://hubs.la/Q01ZZGL-0
💼 Get started in the world of data with our top-rated data science bootcamp: https://hubs.la/Q01ZZDpt0
💼 Master Python for data science, analytics, machine learning, and data engineering: https://hubs.la/Q01ZZD-s0
💼 Explore, analyze, and visualize your data with Power BI desktop: https://hubs.la/Q01ZZF8B0
--
Unleash your data science potential for FREE! Dive into our tutorials, events & courses today!
📚 Learn the essentials of data science and analytics with our data science tutorials: https://hubs.la/Q01ZZJJK0
📚 Stay ahead of the curve with the latest data science content, subscribe t
Watch on YouTube ↗
(saves to browser)
Sign in to unlock AI tutor explanation · ⚡30
Playlist
Uploads from Data Science Dojo · Data Science Dojo · 0 of 60
← Previous
Next →
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Feature Engineering and Predictive Modeling | Data Analytics with R and Azure ML | Community Webinar
Data Science Dojo
Data Exploration and Visualization | Beginning Azure ML | Part 3
Data Science Dojo
Reading External Data Sources | Beginning Azure ML | Part 2
Data Science Dojo
Importing Data, Accessing, & Creating a New Experiment | Beginning Azure ML | Part 1
Data Science Dojo
Casting Columns & Renaming Columns | Beginning Azure ML | Part 4
Data Science Dojo
Scrub Missing Values & Project Columns | Beginning Azure ML | Part 5
Data Science Dojo
Feature Engineering & R Script | Beginning Azure ML | Part 6
Data Science Dojo
Building Your First Model | Beginning Azure ML | Part 7
Data Science Dojo
Run and Fine-Tune Multiple Models | Beginning Azure ML | Part 8
Data Science Dojo
Deploying Your First Predictive Model As a Web Service | Beginning Azure ML | Part 9
Data Science Dojo
Using R API to Obtain Predictions From Your Web Service Beginning Azure ML | Part 10
Data Science Dojo
Using Python API to Obtain Predictions From Your Web Service | Beginning Azure ML | Part 11
Data Science Dojo
Twitter Sentiment Analysis | Natural Language Processing | Community Webinar
Data Science Dojo
Listening to the Melody of the Universe (LIGO Gravitational Waves Presentation) | Community Webinar
Data Science Dojo
David Wechsler on the Impact of Data Science Bootcamp
Data Science Dojo
Andrew Choi on the Impact of Data Science Bootcamp
Data Science Dojo
Microsoft's Software Engineer Shares Her Experience with Data Science Bootcamp
Data Science Dojo
Michael DAndrea on the Impact of Data Science Bootcamp
Data Science Dojo
Data Driven Decision-Making with Data Science Bootcamp: Artem Kopelev's Revelation
Data Science Dojo
Learn the Fundamentals of Data Science: Srinivas Rao's Experience with Data Science Bootcamp
Data Science Dojo
Re-Learning Data Science with Data Science Bootcamp: Analyst's Revelation
Data Science Dojo
Scale R to Big Data with Hadoop & Spark | Community Webinar
Data Science Dojo
Enhancing Skills with Data Science Bootcamp: Sharon Lane-Getaz's Revelation
Data Science Dojo
Ryan DeMartino on the Impact of Data Science Bootcamp
Data Science Dojo
Software Engineer at Microsoft Reveals About His Experience with Data Science Bootcamp
Data Science Dojo
Wade Wimer on the Impact of Data Science Bootcamp
Data Science Dojo
Analyzing Data with Data Science Bootcamp: Hannah Richta's Revelation
Data Science Dojo
Applying Data Science Skills to The Current Role with Bootcamp: Marcos Lacayo's Revelation
Data Science Dojo
Lance Milner on the Impact of Data Science Bootcamp
Data Science Dojo
Deloitte's Data Scientist Revelation: Learning Predictive Analytics with Data Science Bootcamp
Data Science Dojo
Rajesh Patil's Experience at Data Science Bootcamp As an Enterprise Architect
Data Science Dojo
Michael Atlin on the Impact of Data Science Bootcamp
Data Science Dojo
Amina Tariq's In-Person Experience at Data Science Bootcamp
Data Science Dojo
Ceo's Revelation about Data Science Bootcamp
Data Science Dojo
Stephen Miller Describes His Experience at Data Science Dojo's Bootcamp
Data Science Dojo
Kevin Hillaker on the Impact of Data Science Bootcamp
Data Science Dojo
Marko Topalovic's Experience with Data Science Bootcamp
Data Science Dojo
Text Analytics With Python, Cognitive Services & PowerBI | Data Analytics | Community Webinar
Data Science Dojo
Unisys Manager's Revelation: Visualizing Real Time Data with Data Science Bootcamp
Data Science Dojo
Learn Data Mining with Data Science Bootcamp: Ryan LaBrie's Revelation
Data Science Dojo
Vang Xiong on the Impact of Data Science Bootcamp
Data Science Dojo
Data Scientist's Experience at Our Data Science Bootcamp
Data Science Dojo
Alejandro Wolf Yadlin on the Impact of Data Science Bootcamp
Data Science Dojo
Introduction To Titanic Kaggle Competition | Part 1
Data Science Dojo
Learning How to Code in R with Data Science Bootcamp: Priscilla Mannuel's Revelation
Data Science Dojo
Andrew Berman On Why Data Science Bootcamp Is Better Fit for Him
Data Science Dojo
How To Do Titanic Kaggle Competition in R | Part 3.1
Data Science Dojo
How to do the Titanic Kaggle competition in R | Part 3.1
Data Science Dojo
Delve Deeper into Data Science with Data Science Bootcamp
Data Science Dojo
Bank of America Data Scientist Reveals His Experience of Data Science Bootcamp
Data Science Dojo
Shaena Montanari on the Impact of Data Science Bootcamp
Data Science Dojo
Types of Sampling | Introduction to Data Mining | Part 12
Data Science Dojo
Sampling for Data Selection | Introduction to Data Mining | Part 11
Data Science Dojo
Data Aggregation | Introduction to Data Mining | Part 10
Data Science Dojo
Data Cleaning | Introduction to Data Mining | Part 9
Data Science Dojo
Missing & Duplicated Data | Introduction to Data Mining | Part 8
Data Science Dojo
Data Noise | Introduction to Data Mining | Part 7
Data Science Dojo
Graph and Ordered Data | Introduction to Data Mining | Part 5
Data Science Dojo
Document Data & Transaction Data | Introduction to Data Mining | Part 4
Data Science Dojo
Data Quality | Introduction to Data Mining | Part 6
Data Science Dojo
Related Reads
📰
📰
📰
📰
Beyond the Hype: Why Google’s Open Knowledge Format is Not the "RAG Killer"
Medium · AI
The Kimi K3 Shockwave: How One Open-Weight Model Rewired the Industry in 48 Hours
Medium · AI
10 Milliseconds to Understand What a Shopper Means
Medium · Data Science
10 Milliseconds to Understand What a Shopper Means
Medium · LLM
Chapters (13)
Intro
0:12
What is MDM
1:48
What Problems Does MDM Solve?
3:51
Analytical MDM
6:52
Operational MDM
9:46
Overview of Both Approaches
10:33
Implementation Style
13:13
What Makes MDM Unique
14:16
Solutions to Support MDM
18:24
Need for MDM
23:38
MDM is Changing
31:04
Future of MDM
37:01
Q&A
🎓
Tutor Explanation
DeepCamp AI