Understanding or Memorizing? A Case Study of German Definite Articles in Language Models

arXiv:2601.09313v2 Announce Type: replace-cross Abstract: Language models perform well on grammatical agreement, but it is unclear whether this reflects rule-based generalization or memorization. We study this question for German definite singular articles, whose forms depend on gender and case. Using GRADIEND, a gradient-based interpretability method, we learn parameter update directions for gender-case specific article transitions. We find that updates learned for a specific gender-case articl