GPyT - Generative Python Transformer Model released (the off-brand Github Copilot)

Key Takeaways

what is going on everybody and welcome to the generative python transformer or gpit video uh in this video i'm going to be running through the latest gpit model and since the last time we spoke github actually released copilot and the concept of transformers writing python code has now entered the public sphere in a pretty large way i think it's great all these questions and concerns that that we have and i think to address them we have to do research and to do research we kind of need access to the model and historically the company who calls themselves open ai doesn't tend to release the actual full-sized models so my guess is we will never see the full-size copilot model and even like the gpt3 model is released but you query like you don't actually get the model as far as i understand it so anyway and it's paid so so so yeah not the most ideal research environment um so anyway this model is completely free you can use it and indeed it is really only for educational and research use i would not recommend you use this for any other means um so to use it i have put it up on hugging face which is a popular transformers models hub i suppose um i have put it up in it is in both pi torch and tensorflow so whichever one you use you can continue to use um other than that let's check the config real quick we can see that um main things are like right here we have a context length of 1024 so how much data can we pass as context and then also get back from the model the embedding size 768 the attention heads is 12 and the number of layers is 12. so it kind of matches i believe the small gpt-3 as well as the small gpt-2 i don't think there's actually any difference um someone let me know below if there's an actual difference but i don't think there is um i think that's about all i really want to point out there other than take note that i i believe that github copilot is probably the full-size gpt-3 model so this model is 125 million trainable parameters the gpt-3 full model is a 175 billion trainable parameters so a similar structure in terms of being transformers um but not similar in any way to size i it's just literally impossible for me to train a model of that size even this model i trained on the 150 000 dgx station a100 from nvidia um i only was able to get like two epochs and i can't remember how much cumulative time that took but it took a good amount of time so uh keep that in mind so to use the model uh we come back to the model card and essentially it's just these three lines here i'm going to be running through my i have a write-up that i posted on pythonprogram.net i will link it in the description so i'm going to be just basically taking the examples from here and just kind of showing you what this model is capable of i think it's actually pretty impressive it's probably not co-pilot impressive but it's a model that you can actually play with right now um and actually see how everything is working and there's probably things that this model can do that i'm not even aware of yet so just recently i became aware of like the few shot learning so i tried some few shot learning stuff and i was like oh my gosh even that works like so there's a lot of stuff that i'm kind of surprised um so anyways let's get to it so uh to begin again i'm just gonna be copying and pasting from the write-up you can you can follow along if you want again you if you are on a if you're on a cpu you would just need to remove the two cuda stuff and it'll just be a little slower but you could totally follow along on a cpu and it should be fine um so this is just a simple generate function that i wrote um all that it does is replace the input newline characters with the newline token um and then um otherwise yeah it's just taking the response it just takes the number one response you could you can tinker with that if you want um otherwise just just leave it that's like the most likely to be correct response um and yeah and then it just returns it we'll end up creating a few more helper functions along the way just to kind of show you guys and i imagine some of you guys could probably come up with even better ones that i'm coming up with but we can start by just simply generating you know we're just going to pass import the text import so that's all the model sees it's just import and what does it return to us we will see if this is your first time running it you will first download the model when you run these two lines so it might take you a little longer than me initially but anyways all we did was pass import this was the input import it decided that we were going to import numpy as mp import pi test pandas pd and then import all this stuff from pandas and then start to make this test data frame thing so pretty cool and again you have a context length that is a max of 1024 so um and that's 1024 including the input that you pass to the model uh so we can we have plenty of headroom we can make this max length uh bigger if we wanted oops actually we probably should have passed that here i think the default price should be 100 we'll just pass import uh max length 500 there let me run that one more time okay so now we can see okay there's a little more information here and again we get to this and it starts trying to throw in these pads and so the model is definitely getting a little confused although this is still looking pretty decent code here but at least here right we can take this copy that we'll just come down to a new cell and oh no module name pi test so we don't have pi test um but that would just be a pip install but anyways the rest of this is syntactically correct this is acceptable code so the three years ago when i last kind of played with sequence to sequence models trying to generate python code it was just a struggle to get it to be syntactically correct because so much is going on here right you that i think we might be taking for granted at this point like things advance so fast with machine learning that what was once really cool and impressive is like no longer really people don't care the bar's been raised um but even here we have class it it properly cases the class okay then we get to after the colon it goes new it learns to do a new line it learns the white space indentation it learned so it learned not only the pep 8 naming for classes it learned the pep 8 naming for methods and functions it understands that this is a method therefore we pass self here right again colon new line further tab over that um that's really cool to me that it's learned this uh the comment probably this might even match something almost perfectly in github again only use this for um research and educational purposes and in fact let's go back to the model card and read the considerations this model is intended for educational and research use only don't even trust the model outputs they're not unique it's not yours the model is highly likely to regurgitate code exactly as you saw it so it's up to you to determine licensing if you intend to actually use this generated code the python code was pulled blindly from github so it could be a mix of python two and three among other differences such as tabs being two spaces or four although personally i've seen no instance i don't think is two-space indentation maybe at the end i'll try a two-space indentation example and see if it continues to to follow that structure if that's what it's seen so far um and then finally along with the all the above the code generated could wind up doing anything it could try to dos some website it could try to delete your home directory okay so don't run the code right so again research and development only don't it's just no don't use it for anything else so anyway continuing on so maybe this is like a github issue or something i'm not sure so yeah so pretty cool uh that we that it's at least able to do this but i assure you it does even cooler stuff so uh coming back over so now let's talk about some of like the autocomplete capabilities of the model so i'm just going to come down here paste we'll run this so in this case we're importing matplotlib.pyplot splt we're passing some x and y data and we're just saying okay do what you want from here so we come down here and we see okay the next thing it does is it creates a figure and then goes ahead and makes a plot so we can actually take this also it comments the x next to the y which is interesting um but we'll copy it we'll paste it it's valid code and sure enough we do get the graph right so it at least understood okay you're starting to do some matplotlib stuff you're probably going to make a figure you're probably going to want to make a plot it does do some of this repetition stuff we will address repetition um soon uh but at the end of the day what did it do it created code for us i think i just think it's so cool and the other thing that i'm curious about just live what if i add the new line before we pass it okay so if we add the new line before we pass it it doesn't bother with the figure and it just goes straight to the plot which is still acceptable code by the way um so that's interesting i wonder why it did the why why would it do the comment and like that's why i think it would be really cool to be able to do r d truly on the model um so again this is why i kind of dislike what open ai has been doing lately these are interesting questions why did it choose that rather than um just go you know why did it add that comment that's weird right it's just interesting um so anyways continuing on continuing on we don't have to talk about open ai or closed ai all day uh so the next thing is what if we have that exact same data and rather than a regular um a regular plot we're like no no we want a scatter plot so we actually just the only change that we're adding here is we're just saying okay add a comment that says scatter plot okay so we go ahead and run that and we get again there's some repetition we'll talk about it but we could at least take this copy paste and we get a scatter plot like it again it's valid code and we kind of nudged it to say hey could you give us a scatter plot instead and again we might want to change the colors right like later you might change that to green um the label is i'm not really sure why we don't even need a label but if you want one you can have one or you could just remove you could remove this entirely whoops i didn't mean to do that right so you can do something like that and again it's just cool that it understood the context leading up to this point and then it's like oh you want a scatter plot okay here's a scatter plot and i think what's neat is you know you might like we did you might want to change some stuff and that's why i think copilot is what a great name for something like this because that is exactly what it is like your it gives you some output is it going to be perfect no you might want to make some tweaks but it saves you it could save you time right and i think it's pretty neat so continuing along uh with the commenting what if we said we're going to come out the comment out the y and we just say histogram what's it give us so again it does some repetition but we'll just copy and paste what we want copy paste and again all we did was we passed up to the comment histogram all of this was it just made it itself like maybe you forgot like some of the parameters to histogram so we run this we actually don't need the show but whatever no labels to put in the legend true that uh anyway uh we get a histogram and we get a title and a y label i that's so cool to me i just think it's so cool it's so exciting didn't see that this this can work so again um one of the other things that is kind of interesting is the concept of like few shot learning so we can kind of pass in we can say hey the library that we use for graphing is matplotlib um and in fact matplotlib i'm gonna say dot pi plot as plt like that's pretty much always the right import we could we could try both i'll leave that one and then we can remove it anyway um web requests we are going to import requests for array math what do we do g pi t and we can run that copy come down here paste and obviously the hope is import numpy import numpy as and sure enough we get um import and then it continues to make uh that's funny uh that's funny indeed plot the graph what if we do what if we just do graphing import matplotlib will it go will it go uh no it just continues graphing okay but it does it does do at least as plt anyway what we we in this case all we were interested in was the very next line so one of the other helper functions i came up with was to add this so just do next line only and now if we and all it does basically is it counts how many new lines existed in the input and then it just goes to the very next new line that's it so then if we do next line only based on the input oops we didn't run that cell we see okay for array math we do numpy so again i that is so cool to me that's just so cool to me um so another option that we have is like for example um array math uh we could say uh what do i do neural networks neural networks what does it do it says from keras dot layers we import dense dropout flatten and it really wants to flatten so one more flattened right that's so cool that's just so cool um what if we it won't always work so what if we do deep learning we can kind of try a few deep learning wants to import matplotlib no surprise machine learning whoops right so it's definitely not always going to work um let's see yeah web development websockets i wouldn't recommend it but possible um how about make a website yes absolutely correct it also require imports requests twice but that's okay um okay so again that's really cool yes it doesn't always work i would be very curious to know like um what was it let's try deep learning um yeah i would be curious again or in for just r d and educational purposes what happened there right it's interesting it's an interesting question so um okay so continuing so continuing with the flask idea if we paste this in uh let's say we start to build a website we say okay from flask we're gonna import flask render template we're obviously trying to make an index page a home page what does it do and what it does is it creates at least after this which is basically up to this point it says okay define your index return render template index.html with methods get method only so again accurate that's exactly what we wanted and we couldn't in this case we couldn't do just next line because it would just say define index um so we just went ahead and generated everything but then we also noticed that it does do this kind of like repetition stuff still um so those would still be something you'd want to logically handle for at some point so for example we can throw in this function so at this point if we split everything up and we see that the line is identical to a previously seen line as long as it's not just a simply a new line um then we don't want to see it again so it and then we've reached the end essentially if we see a repetition so uh so for example if we take this copy we come down here we paste and next line oh that's not what i wanted uh generate let's see did that actually so like at least the index here uh we would want so let's do stop at repeat who do this cool so at least that by this point it sees oh you're about to define an index again and it at least stops at that point it might be it'd be enter this one is also incorrect right because we've already handled for index i can't think of a really quick and easy logic way to handle for that um but again as your as your co-pilot it definitely saved you time writing and writing all this uh so at this point you could just delete that or you know do whatever to handle that so next we'll try some good old-fashioned deep learning i took this from my my deep learning tutorial um and actually this isn't in it but the question is okay you pass all of this information can we hint so first off we could just start by doing this and just see what it does um cool so it starts off by take you know just saying hey we're going to start with a dense layer 256 units with an activation of rectified linear which okay that's fine it doesn't actually know at this point what are x and y the only hint might be the com2d or maybe psyphar or something like that but um yeah so that's the next line that it thought would be useful to add what if we don't want that though we want a conv layer so what if we just come up here and we say um like i just said yeah conv we can try other spellings because again there it gets a little finicky sometimes conv so here it says okay we'll do a conv layer we're going to add a comp 2d 64 units then we're going to do max pooling okay and again this is taken from my uh tutorial here so it's possible some of this code was on github already anyways and maybe that's why where it got the it's interesting that it defaulted and wanted to do dents 256. kind of curious about that instead of doing the going straight to the conf but one thing we are missing is an activation so what if for example let's do conv i don't know let's try conflair okay convlayer with relu activation give us the activation i demand it um interesting we really want an activation here but no matter what it wants to see seemingly do this convlair um 128 features i'm just curious nope we tried we tried um okay so convlair let's just continue on we don't um i still want i wish there was a good way to slap in the activation here but whatever we'll we'll tinker with that maybe in a moment so then i'm just gonna follow what i have uh written already for now uh let's say we wanted to do a flatten again we're still missing these activation functions which is a bummer so then it knows okay model add flattened easy enough and then finally what if you're like okay now i want output 10 classes i just think that's so cool um one thing i don't remember if i tested this so i'm gonna say output five classes not quite buddy at least it threw in an activation function though i put two classes tank it wow that's crazy that it knows output 10 classes but none of the others huh another time i saw that um i did i just can't remember the exact pattern that i did to get it but i did like i did uh flatten like i did flatten comment like this and not only did it add the flatten it did a new line new line comment space capital d dense uh and then it added the dense layer i just i had to walk away for a second i just thought it was so cool again i recognize again we're missing the activation here um it's not perfect it really isn't um but it is it's so cool to me and it's again a huge huge improvement over what we had three years ago actually i want to do this um so then finally what if we add optimizer going a little bit off of script here i'm just curious sad oh the mac it might be a max length issue hold on let's do this but then also let's see would it be a max length issue should be fine it just seems like it's more than um optimizer uh model compile see if any of these work sad um what if we do just this let's get rid of that like other messy info really maybe it's trying to throw in some other uh [Music] it gets pretty funny i'm sorry um okay so not the best not the best like i said research and educational use only so uh finally we have some pie torch we can definitely give pie torch a little bit of love so we'll paste in some general pie torch code so essentially we've defined our neural net also can we just i mean this is why i hate pytorch it's like so much code is required to do like this right you got to come down here and you got to do this it's so ugly anyway so for the forward method it does the flatten here and then it does the relu uh does it twice i'm not actually sure what that's going to cause but essentially you would really just need to delete this and you'd be good then it goes to define the neural network again which is kind of weird but actually this is very close to being accurate and again for writing pi torch code i like pytorch because you can do some really cool custom stuff with it easier than you can often do it in tensorflow but gosh it's such a pain writing everything that you need to write so this would be a perfect example why you would want to have um you know something like this to to help you but anyway i think that's good enough for now uh like i said the model is there for you guys to play with just at least for what i have personally found it does some pretty good next line auto complete it can do some pretty good next few line autocomplete uh and then it can also do some very interesting few shot learning stuff that i'm just starting to figure out like how to how to do like an inference pattern and get it to to figure out what i want and to just continue going so it works really well i think with the imports it would be interesting to see it work with maybe one liner functions stuff like that so definitely a lot of room for research development definitely a lot of room for questions as far as how to use these things going forward and what do we do with this technology uh going forward but we really can't answer those questions without having a model to actually test and do r d on so i hope that people will use this for that purpose um and and again i just will remind please just don't use this for anything else it can really it can put out it can output like anything so um just be careful like it's purely for educational research use only um but let me know what you find i feel like every time i sit down and kind of just tinker with this model for like 20 30 minutes i learn a new thing like the last thing was the kind of just trying to play around with the few shot learning and i was really surprised like the you know use doing few shot learning kind of in the methods that most people do with language like did not work and then i was like what if he just used comments to like hint like here's a comment do this thing here's a comment do this thing so without fine-tuning it to like i thought you would have to fine-tune it to do that um and i still think you could take this model and you could fine-tune it for exactly that like you could fine-tune a doc string to a function um completely possible haven't tried it yet but i'm fairly confident that's going to work especially after i've seen doing the uh the comment to code or comment to import at least works and then comment to some code works like we saw with like some of the comments with um you know the layers of a neural network and so on that kind of that kind of works it'd be interesting to see how big of a scope can we get that few shot learning to to work on but definitely i haven't really done any fine tuning of this model so this is just the base model yes there's some cool stuff that we can do right out of the gate but it'd be interesting to see what more it can be done so hopefully some people can do some really cool and interesting things with us and help us to answer some of these really pressing questions that we have um especially now that copilot is out and on the way like like things are about to change i think uh pretty considerably so um very interesting stuff hopefully you guys enjoyed leave your comments questions concerns whatever you know feel free to leave them down below otherwise i will see you guys in another video you