George Hotz | Programming | Context Tree Weighting: compression = AI | Part2

Key Takeaways

tree waiting it's really hot I think I might take my shirt off that's appropriate you get banned from twitch for that can't girls take their shirt off with which saddle out but we're gonna do it we're gonna do it cuz I was out all day I'm not gonna wear a shirt for this stream I don't know what that means I don't know what that means but that's what's gonna happen all right wait so I'm when you're in the chat is anybody here I don't know this really isn't great oh here we go yeah flex no no this is I mean you know I'm into handsome men as much as the next guy but I'm gonna talk about that so first we're gonna go over the drinks we have here we have a 100 plus I heard from my date that this is an isotonic this I I went on a date no I know bang no bang um I went on a date with an advertisement like this girl was we went okay so first off we didn't meet last night because of drama not my drama and I met her this morning for coffee and we walked around the mall this girl was an ad like like I don't there's gotta be something yo you guys ever see ads on the internet and you wonder who clicks the ads like I just don't know who clicks the ads I found them alright no I don't talk about that stuff that stuff that stuff's not what the girls are interested in I talk about traveling talk about other women girls like that who date's an ad that's what I'm all right so that's that um what else do I talk about talk about high school that's just cuz I'm emo alright so today we're drinking 100 plus an isotonic drink and of course amazingly available in this country same cream cheese I was drinking in Japan they're actually kind of better I only export like the high-end stuff which is crazy to think Japan but oh right there in geohawk configuration on github I also committed twitch cgw to github so if you guys want to follow along at home you now can so first thing I'm gonna do is gonna look up suffix trees this is cuz this is like a common thing hundred-plus why does where stream shirtless tonight is this not cool this Malaysia man this is this fun is chilly I don't worry hundred-plus what did I say good morning it's not sleepy time it's one 7:30 look at my computer right up here all right oh it's not 100 plus it's just hundred plus I say okay so I mean so I looked over this paper a bit more this morning it turns out that this paper describes things that are incredibly simple in the most complicated way you can believe like this denotes the probability right but all this stuff says is that it's between 0 & 1 and it's a real number like everybody knows that about it probabilities you don't have to say that right and then like this whole thing has an integral in it and you're like what the fuck's an integral I don't know but then you realize that it's actually just this so when you realize it's just this you realize that we have the wrong numbers here so yesterday I had these set at 1 & 2 and you'll see that with 1 & 2 let's just see that with 1 & 2 I get 50 200 bytes of entropy but then if I use that cough geek romanov estimator it goes down to 5,000 so if you guys remember get a gzip is down at 37 angry yeah I mean I don't think it's that like I really don't think these people wrote it to be like oh I'm smart I think that's just like how they think about things we're just sensible I guess I don't know I'm just kind of not used to it yeah mathematical rigor no one should be better at that well how do you learn to be mathematically rigorous guys okay so when you read this whole thing um you realize that it's memory less and all memory lists is just doesn't uh means it doesn't even depend in the past yeah what else piss me off about this the things start from one things start from one I think MATLAB does that too and that's why these people do that okay can be defined with the source Melanie's memory a finite variable or by our market model is one such mom this can be equivalently described by a binary prediction suffix tree so I've read this page I've never actually written the suffix tree but I've definitely read this Wikipedia and seen this banana example multiple times and convinced myself that I understood it but that's always questionable I probably copy and paste the code for suffix trees for the longest common substring problem in composition programming a structure which is a binary tree and a set of real valued parameters within 0 and 1 with one parameter for each leaf node of the PST sure I can crush me because I guess I want to live in the club right we all know that mm-hmm okay a suffix set is a collection of binary strings s is said to be proper if no string and ass it's a suffix of any other string mean s this is complete if every the fuck does that word mean semi infinite talk too much it's just semi infinite oh um I'm gonna look that up what's a semi infinite mean however I'm going to be semi infant wow it's a real thing abandoned in one direction on bamboo another okay again like fine there's a word for that it's called infinite you know a force like in the word again but yeah like we're also gonna we're gonna read the chat a little bit less again because we're being more serious in these I understand it may not appear that way considering out all of a shirt on but that's just cuz hot here silly and I'm in this room so I'm not my apartment anymore which is another tragedy but that was just poor planning on my part 100-plus isotonic drink it's for people with an active lifestyle yeah most people are there me ads and chat I have the AC on don't worry don't worry the AC is definitely on the problem is it's one of those rooms that has the door thingy and they only give you one key you know cuz they know what you want to do is the door thingy that switches the power so I just got home from a long walk I saw monkeys yo yo guys know I'll shop I showed you the monkeys I'll have to close chat seven can't show you the monkeys but well like it's not like I saw monkeys in cage they were just like monkeys chillin by the street i sat there for a bit like we talked no you can't put literally any card that's only the cheap ones the good ones have actually gotten better and have attached a string oh and leave it and leave it while I'm gone that that would work I think they're macaque yeah they were little um I wasn't afraid of him okay like maybe we should read a little bit more about the suffix trees but I do think I know how to write this I just have a little helper function called second xl1 DG equals pitch and X next leave Brad equals equals L the old rat let's see what it does no I don't use escape nice control-c Oh should we get one of those things where my keystrokes appear on the screen does that make you guys happy but you're all like bullshit like that I better get more subscribers because you know I haven't talked about this no no no no this is a bullshit thing we only talk about them voltage it's just a serious twitch but we actually tried our best to do this oh my god yeah that's great so we want to do is AB those things to the suffix tree six paths shown to the leaves yeah magic to many of you is this a good point we don't want that um yeah so this public efficient ways to write this I'm not sure if these things are just because they care about efficiency let's try to understand here's banana the sixth path from the root to the leaves correspond to the suffixes okay so that's a that's an an that's banana oh I sort of see we can just write the stupid thing quickly I think so buy the stupid thing I mean like we'll just we'll put these in a tree structure now we'll start with a roof node and then these will go back there and we can increment each one in the tree structure like going back that's right that's right oh man ya know I'd see my I be I be made fun of by my mom competition program in people all right maybe Wikipedia page just bad look let's really understand what a suffix tree is Suffolk streams are given types of a compressed tree while all the suffixes the given tasks oh okay well that's interesting so I was looking at it backwards but because this is supposed to be looked at backwards it's to be proper if no string and ass has a suffix of any other string and ass there's figure 1 here we go so this is figure 1 what about mr. Schmidt over no right eraser I wrote a copywriter a someone's before his dream you know where he treasures are sexy yeah I dated and I want to date with an ad she's man I didn't go on a date last night so went on a date in the morning she lost her ID card on the bus so she had to like chase a bus in her I don't know man it was confusing I don't know what path tracer is but yeah we went on a date in the morning like what do you do on a date in the morning okay so they said some stuff here too that was a little bit confusing to me which like just doesn't make sense that this is how you're doing inch oh by the way I remember I thought about a little more another thing I was struggling with that log to like why this is log to the easiest way to think about it is just like like how yeah how many symbols do you want to store forget the negative right if I'm thinking of two possible values there's two possible values now it can have one point six possible values which that gets kind of interesting but you know if that's the way things work probabilistically then there's one point X yeah exactly exactly and I didn't know I was spending another night here yeah yeah it's just you know you know what always kills game man logistics but also she was mad so there's that too but like I don't know it adds adzes there's some nice-looking ads um she's she's pretty but you know like command it's an isotonic drink I learned have you ever heard this word before guys we had something about advertising it's really funny when it's not targeted to you exactly no one's ever heard of this shit's like some weird Malaysia Singapore shit okay ask her where we dad no sweetie I'm not crazy no I'm not really sure it is chill guys no worries chill it's chill and I said you know what we're also drinking some green tea fresh out of Japan it is it is hot as balls here gotta be aggressive what I gotta be yeah no man go yo look your smoke wheelie UNAM that's fine you want smoke weed in China that's fine don't smoke weed in Malaysia don't smoke weed Dubai does what we do Singapore don't smoke weed in Indonesia people crazy don't do that shit okay okay no no this makes sense my PST is a pair were asses of suffix set and we know what that is and each thing has a probability oh you forgot to tell me that it was a real number and the depths of the suffix n is defined as the max like it does almost seem like I want to flip these around right this seems the obvious bringing drugs here I'm sure there's plenty of drugs and Malaysia prison people are out there you know smoking weed right now in this city and be like ha ha ha ha ha yeah I'm gonna say shit but I don't know I will fuck with that though Cambodia you can smoke weed wheeze legal in Cambodia I believe or at least decriminalized so if you make it a Cambodia hey if we're guys I promise if we make it a Cambodia we'll smoke weed I think is this true it's illegal but is it like really illegal Oh although it's illegal please do not harass cannabis users and businesses openly sell cannabis products to the public yeah okay that's that's that's there that's a good kind of illegal so what read it says about - sorry you guys you're distracting me we have work to do get regular they're happy pizzas hmm the owner paid attack so his patrons could smoke weed and his guests as well as thus Jill some time bars will sell free roll joints how do we get the Cambodian guys this coke everybody smoked weed should do that they're good I have that product good weed wait the pharmacies will sell you any legal prescription drug you'd like no I don't believe that these people always ask for shitty drugs like xanax and valiums this is not true not like people think all this is true in Thailand yeah no I doubt what you think the weeds I thought the weeds laced man I'm sure the weeding Cambodia's Chell about some mad sketch Whedon Berlin once well it's no laced with anything good but like the way you know weeds not gonna be laced is if it's like grows everywhere in the country if it's cheap then I got a lace it no do not do drugs in Cambodia we what No okay this guy had meth and heroin but the bad news is Cambodia is far away they looks like all right so I'm here what if we make it to here and we buy a boat what are we in spell everything them in the cloud well wait but so what if we go here can we buy a boat in Clandon and sell our boat across the ocean to kill her to cambodia cool would that be guys how cool would that be no no I actually I know some people here and those are people in kotha and we could get a boat here procure a boat and then sail it to Cambodia no no no but you know what was swimmable this was swimmable and the ferry still took forever but like if you were in the West Valley National Park you totally could have swam from there to there that looks really small discomfort while okay the other option is we could go around no I looked it up I might be wrong about this this is a site I really like Rome to Rio and the problem is your choices are to fly or to take bus this does not sound like a fun bus but do you really want to be on some eleven in or bus okay well actually that road doesn't look that bad I did some of those long buses and uh they now have GPS world here no no this body alright back to work okay so the only question is like are these strings reversed or not that's what I'm not quite understanding and every suffix set as I said to be proper if no string Nass is the suffix of any other string and that's okay so how do you do that let's say I have one string and then later I have this so that's fine because neither string is the suffix of another drink but like this string wouldn't be okay because that's the suffix of that but then at that rate unless I get identical things unless I literally get identical things I'm not sure how busses okay all right III read your chat scroll back safety of transportation you actually have no idea how safe bosses are bosses bosses are safer than trains which is crazy like like I don't know if I believe any of these boss things cuz I've been on some mad sketchy buses right Aaron Aaron Jack string is the suffix of any other so oh okay compressed just like collapse as a couple knows is stupid okay following or all the suffixes of banana I guess we actually want to generate all suffixes to make the tray and that makes sense we want to add all suffixes of each of those things to the tree you can just pick this tall suffix trees of course it's a tree having more possible suffixes as notes they're in all suffixes of individual text this is shirtless trip God the air conditioning starting to kick and they will push share them I went on a date with an ad I hope she never watches my streams maybe read the source code of suffix chase so we can like generate all the suffixes very fundamental techniques all right this whole thing's kind of stupid the set gen thing maybe this isn't what I wanted we need an algorithm to construct one online no no no she was in Addis and she like shills for products I mean that's really the goal of social media the goal of social media is to turn people into ads they can code advertise to their friends okay I understand what a suffix tree is so here's an N squared algorithm to construct a suffix J well that's not gonna work okay I think it might be time to put shirt back on air conditioning [Music] you know fizzle girl nice okay all right let's checking out the other paper who the other people tell me more so this is the first paper that introduced these things mr. Curtis you asked me the actual stay of the finite average resource subsequently this state is used to estimate the distribution that generates the next source symbol the estimated distribution of using arithmetic coding procedures I read this stuff yesterday the concatenation of binary service laughs starting with zero and going negative empty string was a suffix of all strings so that's like the root note is the empty string that make sense statistical behavior of a binary finite memory tree source can be described by the means of a suffix set s each semi infinite sequence there's a suffix that belongs to us our function Maps semi-infinite sequences on to their unique self attacks and s okay this is started this actually makes a lot of sense since all suffixes nests have a length not larger the D so D is that number of bits thing only the last D symbols of a 70 but it's infinite sequence determines up to 80 second for each suffix there corresponds a probability together all parameter formed upon have it okay actually I like this paper better that's less stupid like I was marking it but like okay so those are the parameters your model that makes sense kind of I mean I guess there's also parameters of the model in the suffix tree itself but all right the tree source yeah this makes sense actually next symbol probabilities okay so it seems that such an actually gonna make sense forget about coding the first couple bits now what we'll want to do is we wanna a class proper and complete well has to be complete as well [Music] completeness this suffix is unique since asses proper completeness I don't know how to get yet okay I mean I guess I kind of see what it is right so if both exist yeah so for the first one the only thing we can add is just the one because not not even so we start out with the set containing and like it's a draw a cheese danish thank you for subscribing I'm still hungry guys I had some some what I have tonight I am the ring just pretty go Bob I'm so hungry I'll have to go out off to get more food so this sets not empty the set right now has the empty string in it Suffolk's function function map semi-infinite sequences on to their unique suffix s and all suffixes i have a length not larger than D only the last D symbols so like we're gonna first write again and just bad actually so you can see like okay if you have empty set that satisfies so you can see how the empty just the suffix set containing the empty satisfies proper and complete the problem is we have to conspire time you know espera time in Indonesia you really knew it was prayer time because arguable which operating systems good is stupid which means I made of this against his suffix tells us that the parameter for generating a next binary digit yeah the actual bla probabilities and resources with the same suffix that are said to have the same model but the parameters can be different okay model and suffix that are equivalent the set of all tree models have a memory not larger than D it's called the model class CD to specify model in this model class by a natural code i encode himself except recursively interesting seems complicated maybe the second maybe it's not all example I love example pseudo a source dad suffix at okay yeah this makes sense the weighted probability is defined recursively on the context tree the way to prove it is it okay so let's understand these probabilities how did they get yes it's not gonna split until you have one of each I should have to track the whole thing at each node I mean that's the naive way of doing it each no other corresponds counts a and B yeah okay so you can see that two plus two equals four and one plus two equals three 1 plus 1 equals 2 2 plus 0 equals 2 ok so that makes sense mm-hmm to back at this paper and in GDS okay [Music] let's start out with node and the nodes gonna have an eight-count to be count so it's gonna have a one count and then yeah so how BIG's is Tregoning at sea we'll see how big it gets okay this dust garbage I get to a certain point with complexity where I can't really explain it out loud anymore I can only kind of do it when at that point and the stuff might be wrong bogus bullshit so I'm better I could probably explain it you can kind of see what that's doing right is it right let's say now he's nice okay so that sugar carefully cut the tree right that number gets to like four times what I do wrong oh what's that so that prints the tree six to nine so now we can use the you know that tape T estimated it doesn't like like try criminal so it's one of those STDs you hear about but you've never actually heard anybody ever had that's what I think about when I think of Katie hmm mm-hmm okay that one I shouldn't want to do about it means it got to the end they were still notes [Music] don't to think about that yeah you know I know I know yeah we Yelp and lasering in Internet sorry was he back the Russians are watching that there would just be a one Oh so you something that said that p1 mm okay cool mr. Witter right this whole thing make sense you speaking Chinese in my chat hmm whose work like it shouldn't hm maybe we do this zero shouldn't make sense like house and you're not even created with zeros I wasn't looking stupid well this was the node values it's writing the ad yeah [Music] okay I mean I just finished okay I guess I'm terminating to roll I mean I'm terminating too late that's better okay we've got some packing ratio yeah look at that all right it seems to kind of work what's up the number best you know the marketing so much enter page how is that doing worse than I would it be that bad from my perspective I have good bandwidth I don't know what you guys are talking about fuck morning from the East Coast you're up early okay so that would mean that these probabilities are literally just doing worse than this off I do wrong no no you got the right thing I'm just missing piece that is a suffix true right I'm dropping frames I've dropped 8.5% of the frames that's not so much frames okay the node ass is called the pair nodes and once the zeros there correspond Kent's bigger than equal to bigger than me was here so those are my counts up here see for any children's yeah so that should pretty obviously be true if any look this code right you can write a test for that but I think so probability is defined recursively ah yes but the weighted probability is okay so this is what we're not doing my function to fetch is wrong each node okay this is kind of confusing but this is this is the track right so right now we're rehearsing all the way back and we're just throwing out the other probabilities so this is the node probability so what if we even do this let's just try and the why this has P ones and PC arrows here oh okay it's not totally shitty anymore surprisingly close to 0.5 okay so let's find the definition of PE in this code p zero zero equals one that doesn't make sense I'm using the to each node s NTD we aside a weighted probability which is defined as ya the only weird thing I do there is actually I just want that and say just call get P again I don't know why it should be special alright so then this is just px there oh love it compressed a little bit I'm not very good yeah daddy compresses like the same reason I get more context we're getting somewhere she seems like it should be much better than that I don't understand kaby bugs I think just good okay this just proves something's true about this x NP okay there's no weighted probability I understand why is it true that one - that the treat of that symmetric my only question nice toupee let's just try some simple examples we have like one plus point 5/1 bus yeah it's gonna be that okay that works that's right like that might be the problem changes nothing I said change nothing how many ones in zeros okay so it matches for one over two not my shirt is worse for tip this should match it perfectly doesn't and she does the same as one she's weird into some bug because those two should be the same wasn't supposed to be a morning date dude I don't know man girls know the parts of the world or flaky girls Malaysia like the flakiness is just different that's that's that's all we're gonna say about that that's all I'll talk about about that topic yeah why was work [Music] you it seemed like when are you checking which one it will widest we're doing this a little bit differently you see what yeah okay so we have three backs so I think yeah we have like look up pretty backs or something so let's get rid of this set check garbage because they kind of hate it tinder tinder gold member of Malaysia let's just copy the same code I mean it should be using top right I'm gonna need to make that zero it's a syntaxerror do not have an except yeah I got a slightly different number doing that oh I guess just cuz of the first so you're crazy the best let's just try a little refactor on this [Music] that's the same I don't really understand how those are different domain what else that's three notes call get P to compute that so I'm gonna use it because both though should be filled in there's bug when that should literally be the same data structure as this just represent a little bit differently I'm having an effect I don't understand this is there gonna be same go to this okay that's going a tiny bit off it really doesn't make sense is maybe this why is this so far awfulness called it myself extreme those two dollars should really just track the same bed evident and let's just make something it's got a stupid mind Swan thing which is heroes when we go the stats off a tiny bit oh okay well that explains that so number of bits I think in this case has a different meaning when this says use one bit it actually means use one bit of context so what used to be called one bits actually called zero bits so here we'll say you know just remember that's plus one and this could be no that's - [Music] okay so that's much more similar you know that's tracking way more statistics you can see like this one's the same as this one so these are the same yeah seven notes okay that makes that so you get four there okay that makes sense I don't know why it's different but all right that seems correct so there's stupid recursive things seems worse - I'm a bit - yes and why is that so different leave that there oh look at that that's the best we've seen yet this is next cool the magic of suffix tree okay so I don't understand why they're doing that context waiting thing but this seems to work better if we have context use it yeah look at that look at that new record I let's go up to 16 let's add back in the printing I shouldn't deleted that hate when I delete stuff that later want I'm at an isotonic drink almost I've had a green tea and the only good thing is that the compressor kind of works yeah I know I know I wrote this before get history course not tell I have an old get history Oh guys guys did we just beat dude did we just be gzip we just be gzip oh oh hello this shit crushing it bros absolutely crushing it bros look at that look at that 35 bytes of entropy oh yeah go we go more will be even better Oh Bros Bros that was the easiest shit I love is gonna be hard I don't even use their waiting shit I got it all the way down to pay all the way down sir no ma'am now to be fair the only way to really know whether a compressor has bugs in it or not is to actually create the compressed file and then decompress the file right because if you don't do that you don't actually know if it has bugs give me one sec you know what guys here's what we're gonna do I'm gonna run out and let's see what foods around here and then we're gonna come back we're actually gonna write the arithmetic coder uhm but yeah we'll just say dat ratio bro so yes we've written a better compressor than gzip and how much lines of Python now know better is an interesting and well this is all crap we don't use that so in in 70 lines of Python we've written a better compressor than gzip using context tree waiting now the reason why better is a little bit subjective is because I'm sure it's much lower than gzip gzip is not really designed to be great compression gzip just wanted to be fast I'm good I lick the paper just search for context tree waiting maybe maybe I'll throw that out I'll put a paper therapy and then you can you guys can go my github yes it's just there's a lot of stuff that it Wow a lot of people have implemented this I guess I didn't even know Dan just search for contracts tree waited miss lots of stuff yeah we'll be back we'll be back and maybe give me give me give me ten minutes twenty minutes and then we will implement the arithmetic code hopefully oh yeah such good compression such good compression growth I seen a bit ma'am it was good it was good I had banana roti and it came with curry but it was weird to dip it in the curry so I didn't really dip it in the curry I kind of wanted to get some tosai to dip in the curry but like I stood there with the menu open and I kind of just got the vibe there like who was this guy why the fuck is he into a restaurant so that was kind of the vibe I got so I'm gonna wash my hands cuz I touched money and I ate food touch my life laughter you know oh I got this this was 1:30 Riggins it's kind of a ripoff usually they're a dollar but the roti was a great deal you're gonna write an arithmetic coder oh yes what's your awake I got some technically you're into entropy decoder ah so let's see if we can do this I'm a little bit tired but we'll enjoy our best how long they've walking some monkeys today and I don't mean Australians and I mean like actual monkeys sometimes yo you like what Australians go out to party and drink while monkeys been let's like pick like a reasonable number that's like 10 is 10 a reasonable number okay um so the basics of how arithmetic coding works broken No guys is not racist cuz I'm Australian and a good good good should we go there should we go we talked about how according to some new definitions you're not you can't you can't be racist against white people I'm not nuts Hanna believable but you know now but you have you ever seen if you've ever seen a group a group of Australians when they when they come to like one of these islands to party and then they were in the hostel that just just just crazy it's for a while racism was acceptable only if it's funny I like that snot funny you can't do it that was funny okay so the way that it works this is even more complicated because I also thinking about what if you're wrong there but wait so this is P of X so that's nice we actually need P of 0 hmm it's not exactly right that's P of X Q 0 and P of X so what we actually need is the probability range that it's in so if X is 1 the range is going to be from P of X to 1 else the range is gonna be from 0 point 0 to P of X now yeah this is this is this is arithmetic oh yeah the idea is you just put that range you scale that range by your current range and then once things get too small you have to you like to have to quantize the probabilities hmm quantize them anyway for those of you who missed the date thing you missed the date thing you got to watch the earlier stream you don't get to know if the date was discussed in detail on the earlier stream yeah this this is this hmmm it's cute so this is just normal arithmetic coding there's nothing special about this like I do have I have probabilities I have a model which it cold the definition are starting to work well guys want to see outside I'd wear my denim jacket this morning okay [Music] approximated by 8-bit precision first thank god there's patent hate pants all right so let's round P of x equals P of x times T six doesn't matter oh yeah so it cost us two bytes if we round to the nearest 256 so isn't there a python class calling for action which I go to Python 3 details for action from fractions and port fraction loves fraction no math tool can take oh great it's very different so let's just try it a class encoder so you like just a little bit of encapsulation here what goes here yet say code self px and we're gonna do inc dot code this [Music] fraction as in you know I got a fractions water bottle left a fraction is green tea left zero fraction which is a tragedy I do really enjoy those grantees and I got no fraction of that left either 100 plus more like zero I drank it all right what just happened that was freaky okay so we want to rescale it to self-doubt hi - self dot low and what a multiplier has if that's X is one do that tells we set the height of that it's not right just cold [Music] well you don't actually want PDX there we want P to the zero I know I should just put this finite precision stuff in here there's arguments that I shouldn't whatever those are later on [Music] wait a second this isn't right does it know don't pass an axe this isn't even right hmm see that's what I mean you don't know if you have a working compressor till I'll put this back to pee [Music] yeah you can write that right no that's not right we want to ask us about the first part of the for action I'm still I'm still confused about my great thing you might not even work you know this one works the track is really huge numbers so what do we actually want to say in south dot h let's just try this well those numbers get extremely small how'd it get negative just crap oh [Music] that's better so yeah once we have bits that we can remove from both the fractions so I'm not sure exactly what that means in practice but that's like what the idea of arithmetic coding is right like when your range vision that can be sent to output range after renormalization the normalization no no I mean I think it actually works he's fracas at infinite precision just really small PSEG CDs every time you understand what I was doing wrong there you have an idea [Music] okay I think this is actually a : alright smoke got out arms to t sex [Music] can I say dot numerator is that it I'm probably writing this quite stupidly [Music] the date has been extensively discussed in your division results too large for a float do that still popping fog I'll time I don't even know I'm coming tired it's really slow turned into a mourning day because of aforementioned flakiness this is really slow I think it's right but it's really slow okay I'm not using the fraction plastic fuck fraction no fraction that hmm you know what occurs to me that with finite precision you're sometime gonna get things on the bounce I have some bad news that number before it might be wrong cuz I'm not using X like I don't know how this is this isn't actually returning the probability of X what is it return what is that returning yes that's not right cuz we don't use X right like here we use X and this is correct half a point is about getting excited bros that's it you know what twitch doesn't get to hear about dates anymore oh jeez if I start to tell you about a date just to remind me that Ritchie's don't know about dates anymore this is some of you guys talk about that and like all you're using a Mac or can I get your dot file or what keyboard is that band not they're not that harsh okay that's just the previous so maybe we do this it breaks it entirely it's weird there was matching it just more confident so you didn't return now very rare occasions actually it gets the next one right they don't charge you more if you use more power oh well no I broke it I don't understand him I have what I had before was even close to right but it was producing right answers which is weird yeah I should bring a Bitcoin miner with me everywhere I go I draw like 10 kilowatts well I still think about that if I could pull 10 kilowatts out of power outlet I'm using just the most worthwhile I mean you get yourself thrown at a hotels but would they know you got to be able to take a lot - he yeah well so here's our 10 kilowatts even close to feasible yeah I'm probably not realistically you're probably on like a 15 amp circuit to 20 volts in these parts of the world so you're getting mingler tool kilowatts powers like so you're making 20 cents an hour no it's not so worth it yeah well not really the better thing to do would be to figure out how to like wire into some like streetlights oh it was right I was just predicting for the previous body which probably cheats because it probably uses we crash more than like normal Oh your pecans are worthless well subhanAllah leave damn even tethers are cheap cheap tether tether press I guess it's assuming none of the predictions are wrong some of them are I don't know it's annoying when you think you're close to something and turns out that it was lies okay so that's one that's the same as this thing returns when the answer is born very close to the same thing we go back to the pitcher and those obviously doesn't work okay so that's giving me the probability of the next yes so what I'm not tracking correctly here is like is it the current symbol or is the next symbol this is all wrong man thought we had good shit before turns out was all wrong shit yeah technical analysis is what you don't use like your thoughts about the stock you only use like market fundamentals like Oh what's a p/e ratio shit they all do these people who like pay money to be in these groups of these kids who are on Robin Hood and they're like oh I got day trade of a day trader bro I think that's actually better see you know you just never know that you have your compressor right until you can press a file [Music] that's more correct well we're not being gzip anymore guys these are totally good I think it is crack now that doesn't make sense okay it's install the actual blocker okay so it's all just next symbol probability [Music] oh okay that works okay well now back to the drawing board how about making things better than Jesus is now chitty-chitty maybe now we can use that equation that doesn't work okay now so I feel like we're I feel like we're like the reason this is wrong as we're doing it like the wrong way yeah so that that's of course gonna probably close to that because the most here's to us just being thrown away there we go the other way it's more correct right you see what I mean by the other way okay we're back again worse than Jesus don't don't we celebrated twirly guys we celebrated too early no just great tragic now we're gonna make the whole compressor actually work maybe not tonight but at some point oh okay that's not right but what we really want to do yeah so we're recursing back through things and we want that one to have that point five wait not the current one so I could return the whole list and then compute it afterwards or is there just a cute recursive way to do this mr. Schine West we're constructing a stupid list every time yeah so that's effectively that that should give me like nineties yeah now we want to do is make the first one weighed the highest so maybe just for accent now we're at equals out so to be clear the money alone is the one that goes yeah so we'll get rid of that one put them around rat equals super 500 plus 0.5 times Oh all right well we're not back to beating gzip but at least we're gonna be ambitious and we're gonna write the encoder tragedy but we're actually doing the waiting maybe we can even make this big oh come on Bee Gees up again no I can't be Jesus Jim juice you're the good guys Jesus Jesus super talented you know oh just go Yosh going a huge number notes and doesn't even look better garbage thank you for the bids Dortmund I always appreciated money you know in this world we've all money trust me Malaysia money it's beautiful beautiful morning alright I think we have to settle for just fixing it no no they already got my stream I'll think about the encoder event yeah so okay okay we prematurely celebrated in the previous stream but we really did get things down from like what I ain't wrote in the reasoning what they wire I just dropped telling for BT that Vizag Oh that's it does we got Mavis gym tomorrow she write the ACMA decoder entire dogs bedtime late that was later last no no I'm tired no no that was a lie that was a lie I messed up the calculation that that was that was assuming none of my prediction wrong that's the hypothetical maximum with the levels of precision we had we've actually only gotten to here now we weren't quite predicting the past we were just saying none of my predictions would be wrong the tree was the choosing cutting their own thing it's encoding the right thing now I think I don't know crystally wrong the only way don't know for sure is well write an arithmetic encoder and will generate the file and then we'll decompress it and a little difficulty make sure they're the same stinking okay at least now we have a good well when you talk about domain-specific compressions is why you also talk about the length of the coat and sure I mean everything you can think of is probably not everything like I guess like cheese it's not but like PNG PNG is just Jesus all these 77 zits deflate it deflate the same thing as gzip I don't even know yeah so somebody it's not Jeezy mpg I believe PNG actually uses a Huffman code and not an arithmetic code and this has been talked about before but one of them because Huffman codes are much easier to write he's gonna write this arithmetic coder hmm I didn't try to simulate infinite precision hmm oh gee this might be okay what if I just send one bit at a time to the output in a way it's slower but we want to try to use for action again surely p0 I think it matters but the only thing about p0 is not as I said make some sounds but yeah I mean this is fundamentally what you want to do whether use fraction library or not you see like so what that's doing I'm tracking a low and a high and like you're in the yellow arrow doing that yellow arrow you're in that yellow area but I think I can say if Danny output zero [Applause] and I want the same bug that ever the last time I wrote this I'm sure which was you actually had X really was one let me move that up - there and I've actually was heroes and I think reaction was I won - here it's amazing any code I write it works so maybe I'm a terrible programmer like I don't know people like think this stuff out more well they just kind of try things until it works where people like sit down with like paper and try to figure things out maybe I should do that instead of trying to keep this in my head that sounds boring yeah so then whenever we can remove precision from these fractions we want to show it that means a fraction sucks losing as a way to think about things [Music] well the problem really is this multiplication let's find a reason to fractionate right here we'll say self dot d x equal 36 here we want to do less equals t 0 but we have to rescale these to the old so that's the whole problem I wouldn't have to scale them and just be trivial oh god I shouldn't do this and see what you have to deal with like the fact the name so this was easier like it's so easy in theory but then you have to deal with finite precision can I keep the denominator something reasonable when you multiply fractions you multiply the numerators and multiply denominators right good I don't know is that you multiply fractions does that make sense yeah you multiply fractions you multiply the numerators new month with denominators you subtract fractions okay so you're self-taught range range numerator equals subsidy myself that age range denominator which equals D okay then we want to want to rescale p0 to be in that range so we want to multiply p0 times the range numerator multiplied the denominator times the range to nominate it so right okay the denominators are gonna get huge fast but that's getting negative numbers again I hate fucking negative numbers okay so we take p0 put into the range we can just multiply it by the range numerator three fourths so there are five nine eighty three fourths this is the same thing as 9 over 16 okay so now if X is zero then we want to set too high that's okay and think yeah guys I'm tired real asleep we'll do this more tomorrow they'll make it work when guys are watching my live bat and try to make it work yeah later