Webinar – Large Language Models and Public Opinion Research
September 14, 2023
12:00pm to 1:00pm EST
Registration: Eventbrite link
This webinar is free for DC-AAPOR members and students, and $10 for nonmembers.
Computational Social Scientist, Lisa Argyle, will present her Political Analysis article, “Out of One, Many: Using Language Models to Simulate Human Samples.” Dr. Argyle will incorporate that research into a larger discussion of language models and how they’re being used in public opinion research, including some of the decisions that users need to make about models.
Abstract
We propose and explore the possibility that language models can be studied as effective proxies for specific human sub-populations in social science research. Practical and research applications of artificial intelligence tools have sometimes been limited by problematic biases (such as racism or sexism), which are often treated as uniform properties of the models. We show that the “algorithmic bias” within one such tool — the GPT-3 language model — is instead both fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups. We term this property “algorithmic fidelity” and explore its extent in GPT-3. We create “silicon samples” by conditioning the model on thousands of socio-demographic backstories from real human participants in multiple large surveys conducted in the United States. We then compare the silicon and human samples to demonstrate that the information contained in GPT-3 goes far beyond surface similarity. It is nuanced, multifaceted, and reflects the complex interplay between ideas, attitudes, and socio-cultural context that characterize human attitudes. We suggest that language models with sufficient algorithmic fidelity thus constitute a novel and powerful tool to advance understanding of humans and society across a variety of disciplines.
For the full article, Out of One, Many: Using Language Models to Simulate Human Samples
Speaker:
Lisa Argyle, Assistant Professor of Political Science at Brigham Young University and faculty scholar with the Center for the Study of Elections and Democracy Argyle’s research agenda has two parts. First, she is a computational social scientist, where she works on interdisciplinary research teams to apply new computational tools and methods to substantive questions in American politics, particularly around political participation and polarization. Second, drawing on a background in public opinion and political psychology, she studies how and why people talk about politics in their daily lives, both online and off-line. Within both of these themes and in her teaching, she often addresses questions of race, ethnicity, and gender in American Politics.
For questions about this event, contact Shelley Feuer at program.chair.dcaapor@gmail.com.
More information will be provided for other events as it becomes available. We look forward to seeing you at our future events!