Search papers, labs, and topics across Lattice.
This study addresses the issue of mode collapse in silicon sampling by leveraging Semantic Similarity Rating to enhance the fidelity of responses generated by Large Language Models (LLMs) in political attitude surveys. The authors argue that traditional numeric data generation is inadequate, and instead, they propose a text-only response approach that utilizes text embeddings to map qualitative responses to a numeric scale. Their findings indicate that this method not only mitigates mode collapse but also requires minimal calibration, significantly improving response distribution variance.
Text responses from LLMs can dramatically enhance the fidelity of survey data, overcoming the limitations of numeric data generation.
Silicon sampling refers to the use of Large Language Models (LLMs) to generate responses to surveys. It has shown promise, but tends to generate response distributions with unrealistically low variance. We argue that this mode collapse is due to LLMs failure to generate numeric data, and that text responses may be better suited for this task. We analyze whether Semantic Similarity Rating can improve the fidelity of silicon sampling responses when asked about political attitudes. This method solicits text-only responses from LLMs, then maps this to a numeric scale using text embeddings. We find that this method both improves the fidelity of silicon sampling response distributions, and has few parameters to calibrate.
