What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know"

1. 1Estimate instance-level knowledge scores via multi-sampled inference
2. 2Scale the learning signal according to the model's existing knowledge
3. 3Fine-tune the model using knowledge-weighted learning signals
4. 4Evaluate the model's performance on out-of-distribution data to test its ability to say 'I don't know'

AI engineers and researchers on a team benefit from this approach as it improves the reliability of large language models, while product managers can utilize this to develop more transparent and trustworthy AI-powered products

💡 Estimating instance-level knowledge scores can help large language models learn when to say 'I don't know'

🤖 New approach to fine-tuning LLMs: knowledge-weighted learning to reduce hallucinations!