The Softmax Bottleneck: Why Making LLMs Bigger Doesn't Always Make Them Smarter

When researchers scale a language model — more parameters, more layers, wider hidden dimensions —...