Demoing a Large Language Model running locally on my laptop

Patrick Devaney · Beginner ·🧠 Large Language Models ·2y ago
Model: laser-dolphin-mixtral-2x7b-dpo.Q3_K_M Front-End App: https:/github.com/oobabooga/text-generation-webui Device: hp pavilion laptop 15-cs3xxx Ram:12gb CPU: i5-1035g1 cpu @1ghz Download the model from: https:/huggingface.co/TheBloke/laser-dolphin-mixtral-2x7b-dpo-GGUF/blob/main/laser-dolphin-mixtral-2x7b-dpo.Q3_K_M.GGUF Download the latest release of the text-generation-webui. Extract it to your desired directory. You could also clone it from Github. Downloading this zip resolved an issue with gradio not being recognized by the start executable. Now run the start_linux.sh, start…
Watch on YouTube ↗ (saves to browser)
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Next Up
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)