Rep2Text: Decoding Full Text from a Single LLM Token Representation

1. 1Investigate the extent to which original input text can be recovered from a single last-token representation in an LLM
2. 2Propose a novel framework, Rep2Text, for decoding text from last-token representations
3. 3Employ a trainable adapter to decode text from token representations

ML researchers and AI engineers benefit from this work as it sheds light on LLM internal mechanisms and enables new applications, such as text generation and retrieval

💡 A single last-token representation in an LLM can be used to recover the original input text to a significant extent

🤖 Decode full text from a single token! 💡 Rep2Text framework sheds light on LLM internal mechanisms