Testing Llama 3: Did it Pass the Coding and Reasoning Test?

Key Takeaways

this is amazing now we are going to test llama 3 it is a large language model released by meta I've already covered the basics or the benchmarks in my previous video which I will link that in the description below in this we are going to do coding test logical and reasoning test and finally game creation tests such as creating a snake game that's exactly what we're going to see today let's get [Music] started hi everyone I'm really excited to show you about llama 3 and testing it live we going to use hugging face chat and it contains the Llama 3 70 billion instruct parameter model but before that I regularly create videos in regards to Artificial Intelligence on my YouTube channel so do subscribe and click the Bell icon to stay tuned make sure you click the like button so this video can be helpful for many others like you first we are going to do coding test so it's Python and very easy task so in this we are going to ask the Lun language model to create a function to return the sum of two numbers so I'm going to copy the instruction here and going to ask llama 370b now it is generating the code and here is the response I'm going to copy this and going to test this that is a pass next easy challenge find the discount asking the log language model to create a function to find the discount so copying the instruction and asking the model and now it's generating going to copy this code and going to test it here click check and it's a pause next medium challenge virtual DAC creating a function to convert digital to audio going to copy the instruction and asking the log language model and it's generating the code now and it's more detailed with some examples so I'm going to copy this code going to test it here and it is a pass next hard challenge finding the domain name from the DNS pointer copying the instruction and asking the logge language model now it is generating the response going to copy the code and going to test this and it's the past next going to very hard challenge identity Matrix create a function for generating the identity Matrix copy the instruction and asking the large language model and it's generating the code now going to copy the code and then test it here and it is a pass this is really good next going to the expert level challenge this is really exciting that it could pass all these levels ECG sequence asking the lar language model to create a function to generate ECG sequence copying the instruction and asking the lar language model and it's generating the ECG sequence function I'm going to copy this and going to test it here clicking check that is a fail so going to copy the error code and asking it to fix this error and it's regenerating the code now I'm going to copy the code again that's the fixed code and let's test this final try and it is a fail so this is really a good model it was able to pass up until very hard Challenge and it failed only in the expert level challenge most of the open source model will fail even in very hard challenge but this was able to outperform all those open- Source models that is really good now we are going to perform logical and reasoning test so here's the question Natalia sold Clips to 48 of her friends in April and then she sold half as many Clips in May how many Clips did she Natalia sell together in April and in May going to click enter in April it's 48 and in May it's 24 totally 72 that is correct next W earns $12 an hour for babysitting yesterday she just did 50 minutes of babysitting how much did she earn going to click enter and here is the answer 50 minutes is 5 ID 6 of an hour so she earned $10 that is correct completing a new chat window I'm going to ask those two questions together just trying to understand if the launch language model is able to perform the task parall or in the same request so I'm going to ask the same question Natalia sold Clips to 48 and W earns $12 an hour on the same question again but both together I'm going to click enter so I ask both the questions without any answers provided and it is able to identify the two different problems the first first one is 72 that is correct the second problem that is wrong supposed to be $10 so it was able to perform these questions separately but not together I'm going to ask again but going to add thing step by step and click enter so now it's breaking it down to step by step still the answer is wrong for the first question the answer is correct overall I'm really impressed with this model next the final challenge creating a snake game in Python opening a new chat create a snake game in Python click enter and it's automatically generating the code and the code generation is complete going to copy this code pasted the code in vs code now I'm going to run this pip install P game to install the pame package then python snake game. py and then click enter you can see the game is running the snake is running across the board and you can see I'm just playing now voluntarily I'm going to hit this Lake in the wall and let's see how what happens so as soon as I hit in the wall it resets which means it starts from the beginning and you can also see the score at the top Corner which is really good you can even modify this further but up until now I'm really impressed with this model this is going to be a game changer in the open source large language model world I'm going to create more videos similar to this even fine-tuning this large language model so stay tuned I hope you like this video do like share and subscribe and and thanks for watching