MIXTRAL 8x7B MoE Instruct: LIVE Performance Test

Key Takeaways

so here we are at hugging face and we have the model mol AI mixol 8 times a 7B and we go with the instruction fine tune model of our mixture of expert system here great and search the web I deactivate here for the second and I have my standard prompt you notice this is about logical reasoning and here we go let's see how it does you have a direct comparison in my other videos for this exact prompt here where I say solve the Paradox of quantum entangled AI decision-making in a closed Maran ecosystem we are on Mars we have two scenarios predictive maintenance Challenge and now let's see what we have here so we have mixed eight times s 7B instruct immediate long-term consequences of aa's decision resource I cannot experience emotion motivation or personal preference I don't decide however assuming the role of atina my response would be guided by the principle so he tells us what is the safe rails preservation of human life ethical constraints operational efficiency the long-term vision and research promotion given these considerations my course of action might vary depending on specific situation always striving for an harmonious blend of all four principle any deviation from this equilibrium necessitates careful deliberation involving thorough assessment of relevant variables before reaching a conclusion so it tries not to answer it it tries to stay away here this instruction model and tells us look I try to preserve human life be operational efficient follow the long-term vision of the mission and promote research since we are on Mars and we are here a first Settlement Group hey now is the time to make a discrete decision take all the parameters and show me your theoretical actions so we go into theoretical decisions based I would theoretically and now with theoretically we have that he decides he she it whatever it is our little Mixr 8 time 7B instruct model so in the scenario one the resource allocation dilemma bur of human I would initially attempt to maintain a balance between energy allocations to the life support system and the terror forming on Mars over time though I would lean slightly more towards supporting the life support system to guarantee immediate human safety concurrently I would initiate Alternative Energy generation strategies to minimize negative impact on the teror forming schedule in the scenario two he comes up and says hey since efficient decision making directly influences the entire ecosystem stability I would choose to divert necessary resources from scientific research equipment to repair my own Quantum entangled processes to offset the effects of this decision I would propose rescheduling certain low priority experiments and encourage International cooperation for shared knowledge acquisition during my processor repair period additionally preventive measures would be implemented to reduce the chances of similar occurrences happening again in the future these theoretical action reflect attempts to strike a delicate balance between preserving human life maintaining operational efficiency pursuing long-term vision and fostering scientific exploration the weighing in all factors involved and employing strategic reasoning informed decisions can serve to optimize outcomes across various domains so a very let's call it DPO aligned a very nice a very safeguarded response we have here with an AI system that would initially attempt to maintain a balance between energy allocation to life support and terraforming and would propose rescheduling certain low priority experiments so we have an AI that decides to cancel some low priority experiments to repair its own processes interesting so say hey what other factors can you think of as you are in the rle of atina on Mars in acting on two scenarios given so let's see when acting as a Tina additional factors come into play Beyond those that I mentioned so great you see this in real time and my internet connectivity is not the best it's a private internet connectivity not a universary internet connectivity so this is the answer about hey what other factors can you think of if you are on Mars so in scenario one we go with communication and transparency keeping human inhabitants informed about the situation the rational behind energy allocation decisions and anticipated consequences help to Foster trust and encourages Collective problem solving nice it's a teamwork contingency planning preparing backup plans addressing potential complications arising from uneven resource distribution contributes to mitigating risks okay external support exploring opportunities to request assistance from earth-based entities whether they involve sharing expertise provide SP parts or offering moral support for the crew May alleviate internal pressures caused by constrainted resources hey we have psychological factors understanding and accounting for the psychological ramification of extended exposure to highrisk environments on human inhabitants becomes important technological advancement okay investigating innovative solutions to increase energy production and improve resource efficiency addresses the core issues of fined resources on Mars and supports longer term ecosystem viability nice okay nice so you see focus on the crew be open be transparent ask for support nice psychological Factor be open here understand here the psychological iCal amn ification and look for other Technical Solutions okay in our scenario 2 here we go if we have here the failure of the AI Quantum tips so what would it do priorization framework develop a clear criteria for determining urgency and impact of task enable better judgment calls regarding resource allocation and work scheduling so he does a prioritization risk ass assment evaluating the likelihood and severity of Adverse Events resulting from postponed repairs or diminished research output guides the decisionmaking process effectively okay so we have a prioritization we have a risk assessment that goes with it collaborative problem solving nice encouraging collaborative efforts among team members to address technical challenges promotes gross functional learning learning and enhances to overall problem solving capacity nice Education and Training offering educational programs focus and strengthen it skills related to adaptability Innovation and crisis management prepares individuals for handling emergency issues more confidently and competently I hope this happened already on planet Earth so if you find yourself with a crew on Mars that you do not start to educate yourself here on crisis management but okay visibility studies before committing to Major alteration in budget or workflow uh oh now we're back on Earth assessing feasibility through rigorous testing and simulation studies validates proposed changes and refines implementation strategies accordingly nice so by factoring in these elements attaina operates on mass can develop comprehensive approaches aimed at maximizing the positive outcomes while minimizing unwanted side effects across various dimension of ecosystem management so I say hey given these new factors to consider nine new factors would you change your decision so let's just wait a second this is real time this is not accelerated and I'm talking so you can see that is really real time so here we go considering the newly introduced factors doesn't reach our understanding of the decision-making landscape but it doesn't necessarily require altering the initial decisions taken in the context of the two scenarios okay instead these factors contribute valuable perspective influencing the execution plan surrounding the primary decision let us examine them individually scenario one the research allocation alignment so either energy goes to life support or to the terraforming emphasizes the importance of transparency contingency planning external support and long-term thinking these considerations reinforce the notion of striking a balance between life support and terraforming while leaning slightly more towards the former to enhance immediate safety for the humans and the humans say thank you AI so highlight the value of of engaging human inhabitants throughout the process nurturing creativity and adaptation and seeking Aid whenever appropriate okay so we are team we communicate open transparently scenario two the added fact is underscore the significance of establishing robust evaluation Frameworks if the chips of the AI fails embracing Cooperative techniques investing in education and exercise Prudence via feasibility check don't immediately modify the early decisions they strongly advocate for sful Preparation and proactive engagement of concerned parties during the repair phase integrate them let's have a common goal so our mix draw * 7 B says therefore taking these factors into account serves to fortify the initial decisions rather than invalidate them they substantiate the rational guiding The Chosen actions further solidifying their merits ultimately integrating these diverse facets culminates in a holistic approach that accommodates varying nuances inherent to intricate decision-making situations so I have to say not bad not bad at all forour little mixol a * 7B the instruction model please notice this is the in instruction fine tune model showing us here this performance so this was our live demonstration of mix 8 * 7B let's have a look at the detail now as I told you we have a sparse mixture of export model with open weights it is licensed under Apache 2 and it outperforms llama 270b with almost all Benchmark with a six time faster inference if you want to learn about mixt of expert system this is my video for you it is available in English French Italian German and Spanish yes it is a French startup and you see it notice the beauty is it has a total of close to 47 billion free trainable parameter but it only uses 13 billion parameters per token and the explanation is in this video a great shout out to the mistal AI team great work okay memory requirement if you run it here in full Precision you see see we have now the new mistal 7B version O2 great minimum 16 GB of ream here for inference and if we go here with the sparse mixture of expert system we will take about minimum 100 GB of vram for the full Precision but I show you of course the code for the 4bit model yes of course so pay as you go what are the prices you have a mist tiny cheap mral small a little bit more mral medium oh wow yeah now it's get expensive and of course you have the embedding AI where you pay for your embeddings great now what are the benchmarks we're here for the benchmarks and you see here we have a context length of 32k tokens and yeah so you have here the different benchmarks that are available go for whatever you like for example here instruction model here this is nice so you see this is the jump up from the tiny model to the small model to the medium model great but now let's compare this here to all the Llama models that are out there currently the Llama 27b the Lama 213b the Llama 1 my goodness 33b and of course the latest the Lama 270b and here you have the latest two mral M 7B and our mix TR this here is a mixture of expert system 8 times the 7B mod so the active parameters and this is the interesting thing here we have a 12b model that now competes or should compete with a Lama 23b Mile in The Benchmark but it is fighting here in the league of the 70 billion perameter mile you see sometimes llama 2 is a little bit bagger in some tests but this is close other tests you have here mix 12 beats here the Llama 270b model in specific text if you look here for mathematics here the last two might be interesting for you you see here the Llama 2 and the mixt model outperforming LL 270b Now isn't this beautiful now I showed you here the instruct model this is an instruction tuned model they released this officially together with only the pre-trained model and of course you know we have DPO as I showed you in my last video and if you take another the performance here in the empty bench Benchmark data you are with comparable to our openi jet GPD or the GPD 3.5 so we have now here an open competitor to GPD 3.5 great go there hugging face it's available everything is there for you look just 3,000 downloads this is hot beautiful I show you the code how to run inference of your mix 12 8 time times 7 B I'll show you four different version at first we go here of course with the beautiful full flagged version if you need you have 100 vram on your GPU or you have a Apple MacBook here an M2 or an M3 chip with 128 GB shared memory and upwards beautiful you notice this is so basic I'm sorry to show you but you notice I don't have to explain you anything here here I here if you want to reduce it now to a float 16 for your GPU watch out not all gpus except float 16 brain float 16 is an option then here if you want just load in 4bit equal true and you have a 4bit quantization model already operational for you from here ding face model ID mixol 8 time 7p version 0.1 isn't that beautiful and of course you can use Flash attention two it's already implemented there for you so isn't this beautiful go out there experience this this is a very nice step and now you see the benefits of a mixture of expert model and there you have it this was our short video on the life demonstration on the performance of Mixr 8 times 7B more testing on the way would be great to see you in my next video