Rep2Text: Decoding Full Text from a Single LLM Token Representation
📰 ArXiv cs.AI
Rep2Text framework decodes full text from a single LLM token representation
Action Steps
- Investigate the extent to which original input text can be recovered from a single last-token representation in an LLM
- Propose a novel framework, Rep2Text, for decoding text from last-token representations
- Employ a trainable adapter to decode text from token representations
Who Needs to Know This
ML researchers and AI engineers benefit from this work as it sheds light on LLM internal mechanisms and enables new applications, such as text generation and retrieval
Key Insight
💡 A single last-token representation in an LLM can be used to recover the original input text to a significant extent
Share This
🤖 Decode full text from a single token! 💡 Rep2Text framework sheds light on LLM internal mechanisms
Key Takeaways
Rep2Text framework decodes full text from a single LLM token representation
Full Article
Title: Rep2Text: Decoding Full Text from a Single LLM Token Representation
Abstract:
arXiv:2511.06571v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have achieved remarkable progress across diverse tasks, yet their internal mechanisms remain largely opaque. In this work, we investigate a fundamental question: to what extent can the original input text be recovered from a single last-token representation in an LLM? To this end, we propose Rep2Text, a novel framework for decoding text from last-token representations. Rep2Text employs a trainable adapter that
Abstract:
arXiv:2511.06571v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have achieved remarkable progress across diverse tasks, yet their internal mechanisms remain largely opaque. In this work, we investigate a fundamental question: to what extent can the original input text be recovered from a single last-token representation in an LLM? To this end, we propose Rep2Text, a novel framework for decoding text from last-token representations. Rep2Text employs a trainable adapter that
DeepCamp AI