LM-Guided Chain of Thought

Key Takeaways

let's came across this very cool paper that uses a bunch of ideas to improve reasoning in LMS using small language models it first applies knowledge distillation to a small language model with rationals generated by the large language model with the hope of narrowing the C in reasoning capabilities essentially the rational is generated by the lightweight language model and the answer prediction is then left for the Frozen large language M this resource efficient approach avoids the need to finde the large M and instead offloads the rational generation to the small language Mo the knowledge is still language model is for optimized with reinforcement learning using several rational oriented and Tas oriented reward signals the framework is tested on multihop extractive question answering and performs all baselines in terms of answer prediction accuracy reinforcement learning helps to improve the quality of generated interal which further improves question answering performance the AL guided Co prompting approach proposed in this paper performs both standard prompting and Chain of Thought prompting self-consistency decoding also enhances performance the Recon like this paper is the clever use of small language models for rational generation the results are remarkable given that large language models are preferred for disability over smaller ones not everything needs to be done by the large models when fine tuning it's useful to think about what exact aspect you want to optimize and test to see if a small language model can do it for you