Can't seem to enable reasoning in llama.cpp
📰 Reddit r/LocalLLaMA
Enable reasoning in LLaMA models using the correct parameters and prompt formatting
Action Steps
- Check the model's documentation to ensure it supports reasoning
- Use the correct parameters such as --chat-template-kwargs '{"enable_thinking":true}' to enable reasoning
- Verify the prompt formatting is correct, including the use of /think at the end of the prompt if required
- Test different reasoning budgets using the --reasoning-budget parameter to find the optimal value
- Consult the LLaMA community or official documentation for troubleshooting and support
Who Needs to Know This
Developers and researchers working with LLaMA models can benefit from understanding how to enable reasoning in these models to improve their performance and capabilities
Key Insight
💡 Enabling reasoning in LLaMA models requires careful parameter selection and prompt formatting
Share This
🤖 Get reasoning working in LLaMA models with the right params and prompt formatting! 📚
Key Takeaways
Enable reasoning in LLaMA models using the correct parameters and prompt formatting
Full Article
Hi, I'm trying to use some LLMs which I know support reasoning (TheDrummer Rocinante X 12B model) but I can't for the life of me to get it to work. I've tried using all these parameters: --chat-template-kwargs '{"enable_thinking":true}' --reasoning on --reasoning-budget -1 But to no avail. I've also tried adding /think at the end of the prompt, it does nothing. Any idea what I'm doing wrong? Thanks! <!-- S
DeepCamp AI