Ollama vs Private LLM on iPhone: Llama 3.1 8B Edition

Private LLM · Intermediate ·🧠 Large Language Models ·1y ago
In this epic Ollama vs Private LLM showdown, we test Private LLM on an iPhone 15 Pro Max against Ollama on a powerful 64GB M4 Max MacBook Pro. Both are running the same model, Meta Llama 3.1 8B, but with a twist: - Private LLM uses cutting-edge 3-bit OmniQuant quantization. - Ollama relies on 4-bit RTN quantization. The prompts are identical, the settings are the same (temperature 0.7, top-p 0.95, system prompt: "You are a helpful AI assistant"), but the results? You’ll be shocked! Watch as we test: 1️⃣ Reasoning skills: Can both solutions handle tricky logic? 2️⃣ Logical consist…
Watch on YouTube ↗ (saves to browser)
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Next Up
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)