Simon Münker, Achim Rettinger, Damian Trilling
Challenging the Myth: A Research Arc on LLMs as Human Simulacra
Proceedings of The Big Picture v2: Crafting a Research Narrative at the Annual Meeting of the Association for Computational Linguistics (ACL)
When Large Language Models (LLMs) combined with prompt-based approaches as human simulacra emerged, they promised revolutionary shortcuts. Models trained on vast internet corpora may replicate human behavior and communication through text-based alignment. The initial optimism of the NLP community positioned LLMs as universal human proxies capable of replacing participants in surveys, generating authentic social media content, and simulating diverse cultural perspectives. We systematically dismantle this "myth of universal generalization" and document a shift toward methodological rigor. Our research reveals fundamental limitations: LLMs exhibit inhuman response patterns in psychometric assessments and produce detectable synthetic content. We analyze the difference between superficial linguistic fluency and genuine human-like representation, and reframe the current paradigm from asking "can LLMs replace humans?" to "under what validated conditions might LLMs serve as useful research components in social sciences?" Our work shows how interconnected research efforts challenge foundational assumptions and establishes best practices for deploying LLMs as human simulacra.
