The finding
Chain-of-Verification drafts a response, creates verification questions, answers them independently, and then revises the response. The authors report reduced hallucination on several factual-generation tasks. [1]
What it means for pingpong
How does seeing a previous answer affect the next assessment? Our reviewers see earlier responses so they can address their weaknesses. The review frame asks them to assess that text independently. Seeing it can still influence the next answer.
The limit
pingpong is not a blind verification system and does not implement CoVe's separate fact-checking procedure. Instructions to think independently are not the same as withholding a draft from a checker.
Source
Chain-of-Verification Reduces Hallucination in Large Language Models
Shehzaad Dhuliawala et al. (2023)