Does a model's stated reason for rejecting a candidate do any work?

RESUMO

[Abstract partially paraphrased by fetch tool] Asked to choose between candidates and explain the choice, a language model often rejects a rival by naming a fact its profile lacks: no director, no date of death. The work tests whether these stated reasons influence decisions by inserting the named fact into profiles and measuring choice shifts across six open models on 2WikiMultihopQA. Supplying the named fact moves its choice more than the irrelevant control does, odds ratio 3.57 [1.54, 8.26], Holm p=0.0210; placing the same fact elsewhere produced weaker effects, and even irrelevant sentences shifted choices significantly. All measurements were validated as string-based rules, uncovering eight parsing defects that substantially affected reported contrasts.