The finding
The authors show that models can agree with incorrect statements despite having the knowledge to reject them. They also report that a lightweight fine-tuning intervention using synthetic examples reduces sycophancy on held-out prompts. [1]
What it means for pingpong
A pingpong review needs to check whether a model uses what it knows when the conversation suggests a conflicting answer. A reviewer can return to the original task and challenge an earlier answer. Whether that helps needs to be tested.
The limit
Synthetic-data fine-tuning changes model weights. pingpong's review framing does not. The paper's measured improvement cannot be attributed to our prompts or advertised as our result.
Source
Simple synthetic data reduces sycophancy in large language models
Jerry Wei et al. (2023)