Crowdfunded Board Games

Key Takeaways

foreign everybody today we're going to stretch the definition of what should fit inside the data skeptic ad Tech season we're talking about crowdfunding board games I would argue that's absolutely a form of advertising but I suppose crowdfunding blurs a few lines in any event I've been holding this one back for a while to release around the holidays I have a sneaking suspicion many of you are out there spending some of your holiday time off if you've got it playing a few board games not only is that a common Pastime but I have like I said a sneaking suspicion a larger percentage of the data skeptic listeners than the typical population does that maybe one day I'll conduct a survey to find out in the meantime today is an exciting discussion about some of the metadata you can scrape from Board Game Geek and what inferences you can draw about the success of crowdfunded board games and their properties all that more coming right up [Music] sure my name is Johanna Sparks I'm a an assistant professor at the Vienna University of economics and business I'm also a Affiliated researcher at the complexity science of Vienna and I'm very excited to be here and could you share okay we're going to discuss I'd love to know a little bit about what your uh the you know broad sweeping umbrella of what your research contains what sort of things you take an interest in these days I'm intro I'm very interested in how the web and different kinds of organizational forms that it enables uh have impacts on our society on the economy on how we organize our lives for instance something we'll probably talk about is this uh idea of crowdfunding or crowdsourcing another good example is open source software how the the web lets us all work together and create this really magnificent large structures that for the web was around you couldn't really imagine they would have created that's what I'm working on these days um I also have a background in in methods so I sort of a data scientist but I I really like to use the methods of network science and we'll see a little bit about that I guess in the paper studying the internet and uh the kind of emergent communities and things that form around it seems pretty multi-discipline area could you talk a little bit about some of the methods you find useful in your toolbox oh yeah absolutely so um I think that maybe the best way to see how interdisciplinary this area is is to look at the the people I work with my co-authors so I have co-authors my my PhD advisor Janos characters is a physicist my co-author on this board games paper that we're going to discuss is a sociologist and I work with computer scientists I've I've even worked with historians really what we do is because I I think there's an advantage that we're addressing these relatively new kinds of structures that we find on the web basically no one knows exact no one knows the right way and maybe there isn't the right way to address questions about these new structures we do so or we try to do so by pulling together really broad perspectives um as for the methods the the network science is really useful because a lot of the data structures we run into on the web are by their very nature relational so what do I mean by that I mean um things happen between people on the web people collaborate people send each other messages people interact on the same file and you can certainly describe that as a kind of in a tabular way so this so I called you you called me or you sent me an email I sent you an email um but to get a to get a really nice map of the whole of everything that's going on uh the network science approach is quite useful and were you a board game player before you were a board game researcher I should be careful here I I sometimes play board games I'm well past Monopoly so one of my favorite games is Settlers of Catan sure but that's a that's a very mainstream game so I I think the hardcore gamer fans uh might be a little bit disappointed I don't know as much as as many of the the people for example the people riding on boardgamegeek.com I'm not nearly at that level well at what point did you take a scholarly interest in board games the story of this paper is is kind of uh interesting and serendipitous so I was thinking a lot about how we can understand the impacts of this phenomenon of crowdsourcing or crowdfunding crowdfunding crowdsourcing even though it's been around for a really long time it's really taken off in the last 10 years again on the web so they're actually examples from history from the 18th century 19th century of people or governments or organizations using some kind of thing that you might call crowdfunding to solve problems the British government in the 18th century had this this notion of the longitude prize this was a scientific kind of competition where they said whoever comes up with a solution to the problem of calculating longitude at C which is actually is actually a very difficult problem very useful for the British government wins a really big prize in the many millions of dollars in today's currency and this this you could say this is the first instance or an early instance prototype of crowdfunding there are also people individuals like um the famous philosopher or social scientist August compte he was living in Paris and he um had the problem that he was constantly running out of money while he was riding his philosophical tracts and social scientific works and he actually put an ad in the newspaper and I said if anyone likes my work I would be very happy if you would send me some money it's unclear if it worked out for him but he certainly tried and there are some other examples but that there are these solo examples kind of um the exception proves the rule there wasn't really anything like crowdfunding until uh or at least widespread adoption of crowdfunding until the web came around and the web is a wonderful place and there's so many platforms where you can ask people for money you can ask on Kickstarter there are different models of crowdfunding on the web for example you may be familiar with patreon and there are many uh GoFundMe there are many platforms where people can ask for support for their personal lives or for projects they do and they're different many different models of getting these people funding and support I was aware of these platforms for a while and I was always thinking okay does it really make a difference in our economy our society is it really changing the things that are being produced what I noticed in the literature was that there are so many papers comparing outcomes or results within a platform so if you go on Kickstarter you might not naturally ask you see the projects that are raising money which ones are going to make it to their goal there are many many many papers that say okay it depends on the structure of the team or how they present themselves or how big the goal is or how if they have a nice picture or something like this there are papers that study all of these factors and those are interesting don't mean to demean them but what if you're looking within Kickstarter with any within any of these platforms you can't really say anything about how do crowd-funded products or crowdfunded things differ from traditionally created products this led me to to search around for an example of an industry where crowdfunding was prominent but not the only thing you might think of the music industry there are a lot of people who crowdfund music but it turned out that it was they crowdfund their first album or other albums but it turned out that it was hard to get data on that so which album is crowdfunded that's not really available a friend of mine pointed me to boardgamegeek.com which is a uh the the data source of this paper it's a really wonderful website it's basically the IMDb which is the famous website where you have all the information about any movie you could ever imagine it's the equivalent of that for board games so it itself is a kind of crowd-sourced knowledge base where users can add games that they've played and they describe the games they and they fill in all kinds of data and it's a really great data source as a researcher because these hobbies are really intense about games they really love the games and they care if onboard Game Geek it's listed that this game came out in 1999 but they know and they have the proof that it came out in 1998 they go on they log in they correct the data which which for people who use data and research that kind of thing is really great yeah so I think the data quality from Board Game Geek is really high and then I I saw this and I was clicking through the the board games and I noticed that some of them had this little feature that this board game was crowdfunded and then I had a Eureka moment that okay here is a big industry where we have on the one hand a large pool of traditionally published games so someone has an idea they go to a board game Publishing House much like a book or like an a musician might sign with a label they designed the game but the Publishing House takes care a lot of the details and does the marketing does the manufacturing in some cases helps design the game this was the kind of way things were done for a while and then on the other hand there's this new way of doing things uh crowdfunding which really took off so I scraped the data I looked at it and I saw indeed that around 2010 there were just a handful example of games that were listed as crowdfunded by 2016 or 17 it was something like 30 of all the games were crowdfunded so this was a real big explosion that happened quite suddenly so I realized that this is a unique opportunity for me as someone who's interested in the effects of crowdfunding on the product creation process and ultimately what kind of products are released and so what I did is I looked and I started to think about how can I compare games between each other I sat together with my co-author and we started brainstorming and we we realized that each game not only has an immense amount of detail about the designer who designed the game and the artist who made the art for the game and when it came out and things like the minimum recommended age or how long it takes to play around with the game not only do you have this nice metadata but you also have a very special kind of information about each game namely you have their mechanisms so what are mechanisms in a board game well according to board game geek these are these are the kind of Elemental ingredients the things you do when you play a game for example in the game Candy Land you roll a dice and you move your piece around the board this is a mechanism and the simplest games just consist of that right if you've ever played Candyland it's really just you roll the dice you move your character around and then whoever gets to the end first wins it's a very simple game at the time we did the study there were 51 mechanisms on Board Game Geek so 51 different things that a board game can incorporate or not and some games have uh two three four five up to 10 or 11 mechanisms and we realized is that we could use this ontology of mechanisms to describe games and compare them so we can say that if you have a game uh with these three or four mechanisms for example you have a game that involves rolling a dice managing cards in your hand a modular board where you might move the pieces or you might actually move the playing field around and pattern recognition that could be a that would be a form a game of four mechanisms if you remove one of those mechanisms and swap it out for something else you might have a similar game but a unique game and this kind of insight we realize that we have some ingredients to write a nice study we have a nice data set of many different kinds of games we have a categorization of traditionally published games versus crowdfunded games and we have some way of comparing them and from then we proceeded to write the paper and run some models foreign thanks to our annual sponsor weights and biases the developer first ml platform that gives you experiment tracking data set versioning and Model Management their solution integrates with tensorflow sklearn XG boost hugging face and literally every other framework I could think of integrating with your model training process takes just a few lines of code once your machine learning code base or notebook is wired up you've just unlocked the next level of productivity the weights and biases platform collects Aggregates and beautifully displays all of your key metrics and Telemetry so you can track your model training in real time and compare different runs have you ever had to screenshot a matplotlib graph that you drop in an email or in slack to show your boss the results of your work that's like serving a gourmet dinner in a cardboard box with weights and biases all of your collaborators can see the results of different runs with an idealized interactive Suite of tools if you're on or leading a growing team of machine learning practitioners you need to investigate if weights and biases is the right solution for you get started at w and B dot me slash data skeptic and mention us when you request a demo well I don't want to trust that all listeners are board game Geeks maybe some of them even don't know Settlers of Catan uh they just maybe know some Milton Bradley stuff when you look at the games that are crowdsourced maybe making an analogy as you said to music there are musicians that go on big name labels and they get into all the stores on an MTV and there are I guess I'll use the word amateur musicians who tend to I think crowdfund and self-produce their work and it is even though those could be tremendous musicians there is something categorically different there they're sort of like professional and amateur in some sense I mean I'm sure in reality it's a gradient but um do you have the same perception of board games if I bought a bunch of crowdfunded ones would I say these are hobbyist projects or would they seem indistinguishable from the professional glossy titles my my impression and what the the data tells me is that crowdfunded games are are a bit different from traditional games for example we saw that crowdfunded games are much more likely to have a high minimum age requirements so 18 plus or adult mature games you might think of the um the Game Cards Against Humanity Board Game Geek considers a this a board game even though it's more of a social party game it's in our data set so this is a kind of a very different kind of game and and I think that maybe if they had gone through a traditional publishing house they might have had some trouble finding one because it's a little bit racy and loot but there are lots of other ways in which crowdfunded games are different so they're they're more likely to be made by teams of designers and newcomers to the board game industry a lot of designers are crowdfunded games are twice as likely to be made by a newcomer to the industry than a traditionally published game so in this sense crowdfunding is kind of way to get into the industry maybe you have an off the wall idea or maybe you're a hobbyist as you suggested working in your part-time it might be difficult for you to gather what you need to approach a publishing house and they might just reject you because they they don't know you and they they're they might be more risk-averse sure yeah and then the result is I think there are some here we don't have such such strong data but I can easily imagine that the variance in quality among crowdfunded games is probably a bit higher than traditionally published games and we don't have a measure of quality so that you know there's the board game geek rating system but um that's by uh people who are really into games so it might not correspond to to your or the listeners rating of games quality of course is very subjective but my my impression would be that probably crowdfunded games have much higher variance than traditionally published games probably the traditional publishing process kind of compresses things in a conservative way so you've got this pretty rich data set and luckily a nice label in there was it crowdfunded yes or no what kind of research questions can you ask against that data so our our big driving research question was are board games that are crowdfunded more Innovative or new or novel compared to traditionally published games and we had some Theory backing this idea that crowdfunding is a kind of organizational form or even technology that lets like I said before different kinds of people into an industry who might have novel ideas they might have new ideas there is a lot of literature on within the crowdfunded universe of products what's going on what which ones are more successful which ones are coming up with new ideas within the crowdfunding world but there was very little research at least Empirical research about let's compare traditional games versus crowdfunded games or traditional products with crowdfunded products and so we wanted to ask are the crowdfunded products really more new or different in some way and there are a few ways in which games could be different from one another for example they could be distinct from the crowd of Games published before because they combine an atypical combination of mechanisms this would suggest that a game is somehow doing something a little bit different it could also be what we call a novelty which is a combination of mechanisms that up to that point in time had never been put together in a board game before at least in our data set and then finally we have a kind of third take on what it means for something to be a new board game or an innovation and this is the idea what we call in the paper Resonance of a board game so we say a board game is resonant if it makes something new and games after it that come out imitate that game so let's say I make a new game that combines mechanisms never seen before together it might just be that that's a very bad idea it might just be that's a weird game that's a crazy game I take Candyland and I add role playing to it no probably no one wants to play that but naively it would be a novelty what makes something a real Innovation we argue is that if someone makes a novelty and then after that game comes out after that this new idea has been created and subsequent people imitate it and so we asked this we asked these three sub questions of this bigger question the bigger question is again are crowdfunded games different from traditionally published games are they more Innovative or new or novel which would and if the answer is yes that suggests that crowdfunding as an organizational form is supporting Innovation and supporting people doing new things not just doing weird things or or off-the-wall abstract things not just hobbyists who make things in their garage and then they play with their friends and post it on boardgamegeek.com which is fine which is great but it's not a place where a significant economic Innovation or cultural Innovation so what we do is we test these three questions in our data are our crowdfunded games more distinct are they kind of different on average from a lot of the games that came before are they more likely to be novel to contain novel combinations of mechanisms and finally are they resonant meaning when they come out do subsequently publish games imitate their combination of mechanisms and we find that the answer is yes to all three of these questions crowdfunded games are more distinct or more novel and more resonant in this way and we take that and this is a bit of a theoretical step of course you have to follow our argument that's where the data analysis ends in our theoretical argument continues we argue that crowdfunding is a kind of organizational form is pushing the industry in a New Direction it's a very intuitive result I'm not surprised by it I I almost thought that might be it but glad the science leads in that direction but I feel like almost we skipped a step saying something's Innovative or novel I agree those are the properties of interest but it's not clear to me how you measure something like that how can you decide the novelty or the innovation of a particular game let me go into a little bit a little bit more technical detail about how we Define these things so as I said before each game has a collection of mechanisms Associated to it we have 51 mechanisms in total and so what we do is we assign a 51-dimensional vector a binary Vector of ones and zeros to each game where we list the mechanisms in order and the game's Vector has ones where it has that mechanism and zero otherwise and this is not an idea we came up with there's a nice paper by Aaronson and schilling where they do something similar for patents patents have their own ontology they have their classification so a patent for a certain kind of computer might have certain sub-components that you could treat in a similar binary Vector setting anyway we adapted this to the setting of board games and so each board game is a 51 dimensional Vector a binary vector and then what we do is we can say okay here are all the games that were published before this game came out what is the distance the Hamming distance which is a a binary Vector a distance between binary vectors or a notion of distance between binary vectors how far apart is this game from all the games that came before a measure of distinctiveness is just the average of that distance so again it's it's on average how far away are you from this massive games that came before you another thing we can do getting to the novelty part when is the game doing something completely new well then we take the same notion of distance we do is we take the minimum distance so if there's a game that implements the same mechanisms that you have then your distance to that one will be zero and then you're you don't have a novel T So novelty can actually be a binary variable which we say a game that is a novel is a novelty if Hamming distance is at least one from all the games that came before but you could also do this and we have this as a robustness test in the paper you can have two-step Novelties let's say you make a really weird game combining mechanisms that are very rarely seen together and you calculate the distance to all the other games that have come before and the minimum distance is two or three it means you're two or three steps away from anything that's been done before so there's also a kind of more not continuous but count type notion of novelty that we could calculate but in the main regressions in the paper we consider just this this binary notion of is this game a new combination or not and approximately I don't know if you have any high level statistics on like what percentages or novel or just some Trends what are yeah uh what are some of the high levels yes so uh about just around half of the games in our data sets are uh our Novelties so do something completely new what we actually find is that among the the crowdfunded average is around two-thirds so around two-thirds of crowdfunded games or Novelties and just still around 50 a little bit less than 50 percent of the traditional games are Novelties and we find in the regressions for example a crowdfunded game has uh so let me say that in the regression framework we control for a lot of potential confounders for instance crowdfunded games may just be longer and more complicated and they may include more mechanisms by their nature and maybe that's it's not a good idea to describe that as novelty it's just that hobby Gamers may have more patience to sit through a four-hour game than casual gamers but we be controlling for those kinds of factors like how long it takes to play the game on average and how complex it is how complex the users of boardgamegeek.com rate it we still find that crowdfunded games are more distinctive and novel for instance a crowdfunded game is about 50 more likely to implement the novelty than a traditionally published game with these controls they're about a fifth of a standard deviation more distinctive and a smaller effect but around a 10th standard deviation more resonant than traditionally published games when you measure resonance and obviously that's sort of a follow-on if I come up with some cool mechanism and then people start to let's not say copy let's say uh imitate to be inspired or whatever does that inspiration tend to also happen in the crowdfunding world or are the more traditional games adopting these Concepts as well so this is something when we first saw the results that crowdfunded games tend to be more resonant we thought okay well there are more crowdfunded games maybe they are just going off into in a weird Direction and leaving the traditional publishing houses to reiterate on Monopoly over and over again but no what what we did is we looked at the Resonance of crowdfunded games if you only consider traditional games that come next and what we see is actually that somehow the sort of center of gravity of the board game industry is following the center as a whole is following the center of gravity of the crowdfunded games so the crowdfunded games are kind of pulling the traditionally published games in a New Direction do you think this work could be used to design a recipe for a successful game if I tried to optimize for your criteria is that a path to creative success I don't think so because uh well I certainly wouldn't bet on it because I think that even though these games are are influential and people imitate them I think that doing something really new is as in many fields more risky I I wouldn't use this to recommend a combination of mechanisms for a board game designer I think there is a practical message here that some of the reasons why crowdfunding might lead to more Innovative more influential games thinking about that can help all different kinds of game designers so one of the inspirational background readings or listenings I did when I Was preparing this paper was a Google Talk by Matt leacock he's the designer of the game pandemic which is a very interesting game that came out a few years before the crowdfunding craze took off so it's it's a traditionally published game this is a game where the people around the board work together to try to prevent a pandemic from spreading around the world this is uh came out well before the covid thing but um maybe Gamers were better prepared for the covet pandemic than the rest of us but anyway it's it was a very successful game and he in this Google Talk which I highly recommend if you're interested in not just board games but also design and how people design creative products what he describes is the process how he was walking one day and he he had this rough idea he'd been thinking about related things for a while and he had an idea and he went home and he actually Drew and he showed a picture of this in his slides on The Talk he drew out the map that he wanted the board game to look like with a sharpie marker and a very crude piece of paper and he started to play the game right away with his wife I think then girlfriend what he really emphasized was that the game more or less the broad Strokes of the game in the sense that it's immediately recognizable even in that form to anyone who has played the the final version was basically done but the fine details were very hard to figure out and he only managed to figure it out it took him a couple of years to publish the game through extensive play testing so working with other people having them play the game and then watching them play the game I think he says something like it was very hard for him to watch people play the game without saying anything but then when he just sat back and just listened and watched them play he immediately saw the problems and how he could fix them what I learned from that is that really successful games are the end result of really extensive play testing or user testing and I think anyone working with digital products understands this that it's really important to to do user tests and user research I think that to look back to crowdfunding what does this have to do with crowdfunding I think one of the things that's great about crowdfunding is that as game designers are developing their game game and then raising money for it they can still make changes a lot of crowdfunding campaigns on Kickstarter have events where the designers meet the players either virtually or in person or meet with backers and then they play preliminary versions beta versions of the game together and then they collect feedback and this is in the crowdfunding literature one of the main mechanisms that suggests why crowdfunding is actually a good idea you can get before you release the final product you can get really enthusiastic fans to tell you what they think about it and that can make an end result much better while I can't offer anyone a formula for making a great board game I do think that the lessons we learned that crowdfunding does seem to work well give us more confidence to think about the theories of why crowdfunding works well and to use the the potential wisdom from that literature with a bit more confidence yeah the board game results that you have empirically in some sense are very intuitive to me that we would find more Innovation there that people with great ideas would self-publish and it would kind of pull the industry forward do you expect to find that as you look into other Industries or something special about board games I do think that many of the results transfer but I can imagine that actually the effects are a bit weaker in other Industries and it's because I think the board game industry is really really tailored designed for crowdfunding maybe that's why the data was actually available and so abundantly so for example many of the games in our data set were designed by individuals or two people or even a team of three there are very few games that are compared to for example the video game industry made by dozens of people so a typical video game created these days has teams of dozens of people hundreds of people working for many years of course there are video games made by individual designers but they are rather the exception crowdfunding may not work as well in that kind of industry so the the question I think this is an open question and something I'd love to answer in the next few years in what context does crowdfunding scale to bigger organizational forms so crowdfunding for an individual board game designer solves the problem of how do I raise money and how do I build an initial user base to test my ideas but if you're trying to build something where you need 10 people there are so many other organizational problems you have to surmount you might have to pay all those 10 people for instance that crowdfunding may not work as well so this is a very this is that this is an open question and I hope research goes in that direction soon what's next for you in your research I'm still very interested in this kind of idea of the web and the the platforms on the web as new kinds of organizational technology that bring us together and let us build things in a way that we couldn't do without the web and right now I'm really interested in how this works in the domain of Open Source software and there's a wonderful data source so the board game geek of the software world is known as GitHub that's not a Perfect Analogy but there's what is true is that there's an immense amount of data on GitHub where if you have someone's GitHub repo you don't just know here's the code here's who wrote it but you have every little code edit you have the the track changes if you've ever used a Google doc with a colleague you have the track changes of a piece of software going back many years and there are some larger software projects or software repositories that go back 20 years or more that are still very active I think this is very exciting not just because it gives us a kind of chance to look at human cooperation digital human cooperation at really large scales but also because it's very important I think software plays a really important role in our society today there's a really lovely book by Nadia all called working in public the making and maintenance of Open Source software by a stripe is put out by stripe press it's a really fantastic book and it makes the case I think rather convincingly that open source software is is not something abstract and used by only Niche users but really rather that it's a really the backbone infrastructure of our Digital Society if you have your smartphone with you and you go to one of your favorite apps if you dig long enough through the licenses and and legal content you'll actually find a list or you should find a list of the open source software libraries used in that app these open source software libraries are maintained Often thanklessly by very small groups and often individuals who do this thanklessly and have to have to maintain software they don't have to but they do it and this is actually I think a major vulnerability in our Digi in our Digital Society and economy we rely on these people we don't pay them we don't take care of them maybe they don't want to be taken care of that another question we nevertheless rely on them and I think we the research Community needs to understand this better and where can listeners go if they want to follow your work so I'm on Twitter I'm on Twitter at Johannes underscore Vox and I have a home page johanneswax.com I think those are the two best places to find me we'll have links in the show notes for people to follow up Johannes thank you so much for taking the time to come out and share your work thanks very much for the invitation that concludes this installment of data skeptic ad Tech in fact it concludes data skeptic ad Tech I've got no preview of next week's interview because next week all you get is me and a brand new season I'm gonna lay out for you what the plan is I hope you enjoyed this season whether you did or didn't drop me a line and let me know Kyle at dataseptic.com I'm not particularly fast in my replies but I do like hearing from people and now that you've heard the whole season your thoughts would be warmly welcome it'll be 2023 when you hear from me next so have a great New Year everyone [Music]