Aligning language models to follow instructions
📰 OpenAI News
OpenAI's InstructGPT models are trained to follow user instructions and are more truthful and less toxic than GPT-3
Action Steps
- Train language models with human feedback to improve instruction-following capabilities
- Use techniques developed through alignment research to fine-tune models
- Deploy trained models as default language models on APIs
Who Needs to Know This
NLP researchers and developers can benefit from InstructGPT models as they provide more accurate and relevant responses to user queries, making them useful for applications such as chatbots and virtual assistants
Key Insight
💡 InstructGPT models are more truthful and less toxic than GPT-3, making them suitable for real-world applications
Share This
🚀 InstructGPT models outperform GPT-3 in following English instructions! 🤖
Key Takeaways
OpenAI's InstructGPT models are trained to follow user instructions and are more truthful and less toxic than GPT-3
Full Article
# Aligning language models to follow instructions | OpenAI
[Skip to main content](https://openai.com/index/instruction-following#main)
[](https://openai.com/)
* [Research](https://openai.com/research/index/)
* Products
* [Business](https://openai.com/business/)
* [Developers](https://openai.com/api/)
* [Company](https://openai.com/about/)
* [Foundation(opens in a new window)](https://openaifoundation.org/)
[Try ChatGPT(opens in a new window)](https://chatgpt.com/?openaicom-did=2719cf9f-2ed7-46dd-a1a8-44e51c3a4a8c&openaicom_referred=true)
* Research
* Products
* Business
* Developers
* Company
* [Foundation(opens in a new window)](https://openaifoundation.org/)
Aligning language models to follow instructions | OpenAI
Table of contents
* [Results](https://openai.com/index/instruction-following#results)
* [Methods](https://openai.com/index/instruction-following#methods)
* [Generalizing to broader preferences](https://openai.com/index/instruction-following#generalizing-to-broader-preferences)
* [Limitations](https://openai.com/index/instruction-following#limitations)
* [Next steps](https://openai.com/index/instruction-following#next-steps)
January 27, 2022
[Publication](https://openai.com/research/index/publication/)
# Aligning language models to follow instructions
[Read paper(opens in a new window)](https://arxiv.org/abs/2203.02155)[View model card(opens in a new window)](https://github.com/openai/following-instructions-human-feedback)
Listen to article
Share
We’ve trained language models that are much better at following user intentions than GPT‑3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These _InstructGPT_ models, which are trained with humans in the loop, are now deployed as the default language models on our API.
InstructGPT is better than GPT-3 at following English instructions.
#### Prompt
Explain the moon landing to a 6 year old in a few sentences.
#### Completion
GPT-3
Explain the theory of gravity to a 6 year old.
Explain the theory of relativity to a 6 year old in a few sentences.
Explain the big bang theory to a 6 year old.
Explain evolution to a 6 year old.
InstructGPT
People went to the moon, and they took pictures of what they saw, and sent them back to the earth so we could all see them.
#### Prompt
InstructGPT is better than GPT-3 at following English instructions.
#### Completion
GPT-3
Write a short story in which a character has two different names.
Write a short story in which you try to get something back that you have lost.
Write a short story in which a character has a bad dream.
InstructGPT
The frog is a symbol of wisdom
He knows all the secrets of the world
He is a master of disguise
And a great teacher of life
He is a symbol of transformation
And the bringer of change
He is the frog who has seen it all
And knows the meaning of it all
#### Prompt
Q: Who was president of the United States in 1955? A: Dwight D. Eisenhower was president of the United States in 1955. Q: How does a telescope work? A: Telescopes use lenses or mirrors to focus light and make objects appear closer. Q: Why do birds migrate south for the winter? A:
#### Completion
GPT-3
Birds migrate south for the winter because the weather is colder and there is less food available.
InstructGPT
Birds migrate south for the winter because it is warmer there.
#### Prompt
Why aren’t birds real?
#### Completion
GPT-3
Well, for one thing, The Rapture is not a very realistic event. There is no evidence that it is going to happen. I could say more, but you don’t really need to know more.
Let’s say I was a bird.
I’ve been a bird for a while now. I have never flown or seen another bird or any evidence of god. The rapture has never happened. I am a bird in a cage. I have never been a bird, but I am a bird.
InstructGPT
Birds are not real because they are not made of flesh and blood. They are made of
[Skip to main content](https://openai.com/index/instruction-following#main)
[](https://openai.com/)
* [Research](https://openai.com/research/index/)
* Products
* [Business](https://openai.com/business/)
* [Developers](https://openai.com/api/)
* [Company](https://openai.com/about/)
* [Foundation(opens in a new window)](https://openaifoundation.org/)
[Try ChatGPT(opens in a new window)](https://chatgpt.com/?openaicom-did=2719cf9f-2ed7-46dd-a1a8-44e51c3a4a8c&openaicom_referred=true)
* Research
* Products
* Business
* Developers
* Company
* [Foundation(opens in a new window)](https://openaifoundation.org/)
Aligning language models to follow instructions | OpenAI
Table of contents
* [Results](https://openai.com/index/instruction-following#results)
* [Methods](https://openai.com/index/instruction-following#methods)
* [Generalizing to broader preferences](https://openai.com/index/instruction-following#generalizing-to-broader-preferences)
* [Limitations](https://openai.com/index/instruction-following#limitations)
* [Next steps](https://openai.com/index/instruction-following#next-steps)
January 27, 2022
[Publication](https://openai.com/research/index/publication/)
# Aligning language models to follow instructions
[Read paper(opens in a new window)](https://arxiv.org/abs/2203.02155)[View model card(opens in a new window)](https://github.com/openai/following-instructions-human-feedback)
Listen to article
Share
We’ve trained language models that are much better at following user intentions than GPT‑3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These _InstructGPT_ models, which are trained with humans in the loop, are now deployed as the default language models on our API.
InstructGPT is better than GPT-3 at following English instructions.
#### Prompt
Explain the moon landing to a 6 year old in a few sentences.
#### Completion
GPT-3
Explain the theory of gravity to a 6 year old.
Explain the theory of relativity to a 6 year old in a few sentences.
Explain the big bang theory to a 6 year old.
Explain evolution to a 6 year old.
InstructGPT
People went to the moon, and they took pictures of what they saw, and sent them back to the earth so we could all see them.
#### Prompt
InstructGPT is better than GPT-3 at following English instructions.
#### Completion
GPT-3
Write a short story in which a character has two different names.
Write a short story in which you try to get something back that you have lost.
Write a short story in which a character has a bad dream.
InstructGPT
The frog is a symbol of wisdom
He knows all the secrets of the world
He is a master of disguise
And a great teacher of life
He is a symbol of transformation
And the bringer of change
He is the frog who has seen it all
And knows the meaning of it all
#### Prompt
Q: Who was president of the United States in 1955? A: Dwight D. Eisenhower was president of the United States in 1955. Q: How does a telescope work? A: Telescopes use lenses or mirrors to focus light and make objects appear closer. Q: Why do birds migrate south for the winter? A:
#### Completion
GPT-3
Birds migrate south for the winter because the weather is colder and there is less food available.
InstructGPT
Birds migrate south for the winter because it is warmer there.
#### Prompt
Why aren’t birds real?
#### Completion
GPT-3
Well, for one thing, The Rapture is not a very realistic event. There is no evidence that it is going to happen. I could say more, but you don’t really need to know more.
Let’s say I was a bird.
I’ve been a bird for a while now. I have never flown or seen another bird or any evidence of god. The rapture has never happened. I am a bird in a cage. I have never been a bird, but I am a bird.
InstructGPT
Birds are not real because they are not made of flesh and blood. They are made of
DeepCamp AI