Simon Münker

Research Fellow at Trier University

I'm a computational linguist and Research Fellow at Trier University, where I study whether Language Models can genuinely stand in for human participants in research, or whether their fluency masks a deeper mismatch. My Ph.D. thesis tested this across psychometrics and social media simulation, and the consistent answer is: not without rigorous, domain-specific validation. I build these metrics that expose where Language Models diverge from human behavior, and the frameworks researchers need to use them responsibly.

Together with related issues in silicon sampling, including language and cultural biases and the lack of ecological validation, my thesis and discussions with colleagues and co-authors sparked the idea for a research cooperative that formalizes, evaluates, and critiques the use of language models to simulate human behavior. I called the cooperative EVAS: Ecological Validation of Artificial Simulacra. Find out more on our website: evas-research.github.io

Publications

20266 papers

Simon Münker, Nils Schwager, Achim Rettinger

Don't Trust Generative Agents to Mimic Communication on Social Networks Unless You Benchmarked their Empirical Realism

Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL)

The ability of Large Language Models (LLMs) to mimic human behavior triggered a plethora of computational social science research, assuming that empirical studies of humans can be conducted with AI agents instead. Since there have been conflicting research findings on whether and when this hypothesis holds, there is a need to better understand the differences in their experimental designs. We focus on replicating the behavior of social network users with the use of LLMs for the analysis of communication on social networks. First, we provide a formal framework for the simulation of social networks, before focusing on the sub-task of imitating user communication. We empirically test different approaches to imitate user behavior on X in English and German. Our findings suggest that social simulations should be validated by their empirical realism measured in the setting in which the simulation components were fitted. With this paper, we argue for more rigor when applying generative-agent-based modeling for social simulation.

Simon Münker, Achim Rettinger, Damian Trilling

Challenging the Myth: A Research Arc on LLMs as Human Simulacra

Proceedings of The Big Picture v2: Crafting a Research Narrative at the Annual Meeting of the Association for Computational Linguistics (ACL)

When Large Language Models (LLMs) combined with prompt-based approaches as human simulacra emerged, they promised revolutionary shortcuts. Models trained on vast internet corpora may replicate human behavior and communication through text-based alignment. The initial optimism of the NLP community positioned LLMs as universal human proxies capable of replacing participants in surveys, generating authentic social media content, and simulating diverse cultural perspectives. We systematically dismantle this "myth of universal generalization" and document a shift toward methodological rigor. Our research reveals fundamental limitations: LLMs exhibit inhuman response patterns in psychometric assessments and produce detectable synthetic content. We analyze the difference between superficial linguistic fluency and genuine human-like representation, and reframe the current paradigm from asking "can LLMs replace humans?" to "under what validated conditions might LLMs serve as useful research components in social sciences?" Our work shows how interconnected research efforts challenge foundational assumptions and establishes best practices for deploying LLMs as human simulacra.

Simon Münker, Fabio Sartori

Guardrail Vulnerabilities in Open-Source Language Models: Implications for Democratic Discourse and Marginalized Communities

Proceedings of the 59th Hawaii International Conference on System Sciences (HICSS)

The proliferation of open-source Large Language Models (LLMs) presents a complex technological phenomenon with significant societal implications. While these models democratize access to advanced Natural Language Processing (NLP) capabilities, they simultaneously amplify risks for marginalized communities who often bear the disproportionate burden of technological misuse. Our research examines systematic vulnerabilities in guardrail mechanisms across seven prominent open-source LLMs, revealing patterns of harmful content generation that threaten democratic discourse and social cohesion. Through empirical analysis using advanced NLP classification methods, we demonstrate that popular open-source models consistently generate content classified as hateful or offensive when subjected to adversarial prompting techniques. These findings directly contradict the safety assurances provided by model developers, particularly Meta AI's stated commitment that their systems should present balanced perspectives on debated policy issues rather than singular viewpoints.

Simon Münker, Nils Schwager, Kai Kugler, Michael Heseltine, Achim Rettinger

Next Reply Prediction X (NRP-X) Dataset: Linguistic Discrepancies in Naively Generated Content

Proceedings of Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) at the Language Resources and Evaluation Conference (LREC)

The increasing use of Large Language Models (LLMs) as proxies for human participants in social science research presents a promising, yet methodologically risky, paradigm shift. While LLMs offer scalability and cost-efficiency, their “naive” application, where they are prompted to generate content without explicit behavioral constraints, introduces significant linguistic discrepancies that challenge the validity of research findings. This paper addresses these limitations by introducing a novel, history-conditioned reply prediction task on authentic X (formerly Twitter) data, to create a dataset designed to evaluate the linguistic output of LLMs against human-generated content. We analyze these discrepancies using stylistic and content-based metrics, providing a quantitative framework for researchers to assess the quality and authenticity of synthetic data. Our findings highlight the need for more sophisticated prompting techniques and specialized datasets to ensure that LLM-generated content accurately reflects the complex linguistic patterns of human communication, thereby improving the validity of computational social science studies.

Nils Schwager, Simon Münker, Alistair Plum, Achim Rettinger

Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction

Proceedings for the 15th Workshop on Computational Approaches to Subjectivity, Sentiment Social Media Analysis (WASSA) at the European Chapter of the Association for Computational Linguistics (EACL)

Sjoerd B. Stolwijk, Mark Boukes, Wang Ngai Yeung,Yufang Liao,Simon Münker,Anne C. Kroon, Damian Trilling

Can we use automated approaches to measure the quality of online political discussion? How to (not) measure interactivity, diversity, rationality, and incivility in online comments to the news

Communication Methods and Measures

This article explores the (in)ability of automated tools to measure the deliberative quality of online user comments along the standards set out by Habermas: interactivity, diversity, rationality, and (in)civility. Utilizing a stratified sample of manually coded comments (n = 3,862) responding to news videos on YouTube and Twitter, we examined the performance of rule-based measures (i.e. dictionaries), machine-learning classifiers (conventional and transformer-based) and measurements by generative AI (Llama 3.1, GPT-4o, GPT-4T). We present results for over 50 metrics side-by-side to judge the opportunity costs of choosing one method over another. The results revealed strong variation across different groups of models. Overall, our expectation that more modern methods (transformers and generative AI) outperform the older, simpler ones was confirmed. However, the absolute differences between these model groups strongly depended on the measured concept, and we observed strong variance in performance among models of the same group. We provide recommendations for future research that balance ease of use with the performance of automated measurements, along with important cautions to consider.

20255 papers

Simon Münker

Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires

Proceedings of 0th Moral and Legal AI Alignment Symposium at IACAP/AISB

Are AI systems truly representing human values, or merely averaging across them? Our study suggests a concerning reality: Large Language Models (LLMs) fail to represent diverse cultural moral frameworks despite their linguistic capabilities. We expose significant gaps between AI-generated and human moral intuitions by applying the Moral Foundations Questionnaire across 19 cultural contexts. Comparing multiple state-of-the-art LLMs’ origins against human baseline data, we find these models systematically homogenize moral diversity. Surprisingly, increased model size doesn’t consistently improve cultural representation fidelity. Our findings challenge the growing use of LLMs as synthetic populations in social science research and highlight a fundamental limitation in current AI alignment approaches. Without data-driven alignment beyond prompting, these systems cannot capture the nuanced, culturally-specific moral intuitions. Our results call for more grounded alignment objectives and evaluation metrics to ensure AI systems represent diverse human values rather than flattening the moral landscape.

Simon Münker

Fingerprinting LLMs through Survey Item Factor Correlation: A Case Study on Humor Style Questionnaire

Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP)

LLMs increasingly engage with psychological instruments, yet how they represent constructs internally remains poorly understood. We introduce a novel approach to “fingerprinting” LLMs through their factor correlation patterns on standardized psychological assessments to deepen the understanding of LLMs constructs representation. Using the Humor Style Questionnaire as a case study, we analyze how six LLMs represent and correlate humor-related constructs to survey participants. Our results show that they exhibit little similarity to human response patterns. In contrast, participants' subsamples demonstrate remarkably high internal consistency. Exploratory graph analysis further confirms that no LLM successfully recovers the four constructs of the Humor Style Questionnaire. These findings suggest that despite advances in natural language capabilities, current LLMs represent psychological constructs in fundamentally different ways than humans, questioning the validity of application as human simulacra.

Simon Münker

Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations

Journal for Language Technology and Computational Linguistics (JLCL)

Contemporary research in social sciences increasingly utilizes state-of-the-art generative language models to annotate or generate content. While these models achieve benchmarkleading performance on common language tasks, their application to novel out-of domain tasks remains insufficiently explored. To address this gap, we investigate how personalized language models align with human responses on the Moral Foundation Theory Questionnaire. We adapt open-source generative language models to different political personas and repeatedly survey these models to generate synthetic data sets where model-persona combinations define our sub-populations. Our analysis reveals that models produce inconsistent results across multiple repetitions, yielding high response variance. Furthermore, the alignment between synthetic data and corresponding human data from psychological studies shows a weak correlation, with conservative persona-prompted models particularly failing to align with actual conservative populations. These results suggest that language models struggle to coherently represent ideologies through in-context prompting due to their alignment process. Thus, using language models to simulate social interactions requires measurable improvements in in-context optimization or parameter manipulation to align with psychological and sociological stereotypes properly.

Simon Münker, Achim Rettinger

twony: A Micro-Simulation of the Impact of OSN Mechanics on the Emotionality of Online Discourse

Joint Proceedings of the European Semantic Web Conference (ESWC) Workshops and Tutorials

We introduce twony, a micro-simulation prototype designed to show the impact of online social network (OSN) mechanics—particularly recommendation algorithms—on emotional contagion and discourse dynamics. By harnessing the capabilities of Large Language Models (LLMs), the system generates an ecosystem of synthetic, politically engaged digital personas that interact within a controlled social media environment. These autonomous agents emulate human behaviors through in-context prompting techniques, enabling users to observe emotional transmission patterns under systematically varied conditions while circumventing the constraints inherent to real-user experimentation. The prototype implements two distinct recommendation paradigms: a baseline chronological feed and an emotion-prioritizing ranking mechanism that amplifies content based on emotional intensity, allowing the examination of the formation and reinforcement of echo chambers. Emotional valence is quantified via a fine-tuned BERT model, while network-level and agent-level visualizations track emotional cascades and polarization dynamics throughout the lifecycle. The system is architected as a browser-based application leveraging modern web technologies and decentralized APIs. Twony emphasizes accessibility, customizability, and extensibility. This contribution advances the field by providing a scalable, open-source prototype for systematically investigating OSN dynamics, offering actionable insights for platform designers and policymakers seeking to mitigate harmful emotional contagion while fostering healthier online discourse environments.

Simon Münker, Achim Rettinger, Damian Trilling

Zero-shot prompt-based classification: topic labeling in times of foundation models in German Tweets

Proceedings of Student Research Workshop at the Annual Meeting of the Association for Computational Linguistics (ACL)

Filtering and annotating textual data are routine tasks in many areas, like social media or news analytics. Automating these tasks allows to scale the analyses wrt. speed and breadth of content covered and decreases the manual effort required. Due to technical advancements in Natural Language Processing, specifically the success of large foundation models, a new tool for automating such annotation processes by using a text-to-text interface given written guidelines without providing training samples has become available. In this work, we assess these advancements in-the-wild by empirically testing them in an annotation task on German Twitter data about social and political European crises. We compare the prompt-based results with our human annotation and preceding classification approaches, including Naive Bayes and a BERT-based fine-tuning/domain adaptation pipeline. Our results show that the prompt-based approach – despite being limited by local computation resources during the model selection – is comparable with the fine-tuned BERT but without any annotated training data. Our findings emphasize the ongoing paradigm shift in the NLP landscape, i.e., the unification of downstream tasks and elimination of the need for pre-labeled training data.

20241 paper

Kai Kugler, Simon Münker, Johannes Höhmann, Achim Rettinger

InvBERT: Reconstructing Text from Contextualized Word Embeddings by inverting the BERT pipeline

Journal of Computational Literary Studies (JCLS)

Digital Humanities and Computational Literary Studies apply automated methods to enable research on large corpora which are not feasible by manual inspection alone. However, due to copyright restrictions, the availability of relevant digitized literary works is limited. Derived Text Formats (DTFs) have been proposed as a solution. Here, textual materials are transformed in such a way that copyright-critical features are removed, but that the use of certain analytical methods remains possible. Contextualized word embeddings produced by transformer-encoders are promising candidates for DTFs because they allow for state-of-the-art performance on analytical tasks. However, in this paper we demonstrate that under certain conditions the reconstruction of the original text from token representations becomes feasible. Our attempts to invert BERT suggest that publishing the encoder together with the contextualized embeddings is unsafe, since it allows to generate data to train a decoder with a reconstruction accuracy sufficient to violate copyright laws.

TWONy(micro)

A Micro-Simulation of the Impact of
Social Network Mechanics on the
Sentiment of Online Discourse

TWONy(macro)

A Marco-Simulation of the Impact
on Recommendation Mechanics on the
Emotional State of Social Media Users

Work Experience

Research Fellow (Trier University)

Oct. 2023 - Present

Implementing and maintaining Python-based modules for synthetic social media users using language models and evaluating discourse metrics for the EU-funded project Twin of Online Social Networks (TWON). Evaluating language models as human simulacra in communication and decision-making. Managing graduate student research groups in applied computational linguistics subjects. Planning and conducting lectures and seminars in natural language processing.

Research Assistant (Trier University, Part-Time)

Mar. 2022 - Sep. 2023

Implemented and co-authored InvBERT: Reconstructing Text from Contextualized Word Embeddings by inverting the BERT pipeline, published in the Conference of Computational Literary Studies. Implemented and maintained a pipeline to scrape social media posts and match them with related news articles as part of the Social Sentiment in Times of Crises (SOSEC). Planned and conducted an exercise in Python programming.

Course Assistant (University of Düsseldorf, Part-Time)

Apr. 2019 - Sep. 2021

Provided academic support for multiple advanced mathematics and computer science courses, including probability theory, machine learning, and formal language. Developed supplementary teaching materials, conducted exercise sessions, and provided individualized student guidance. Assisted professors in grading assignments and managing classroom logistics.

Intern (University of Düsseldorf)

Feb. 2020 - Mar. 2020

Researched multiword expression identification by (re-)implementing a deep learning classifier in PyTorch to improve the detection accuracy.

Frontend Web Developer (DSC-Medien, Part-Time)

Oct. 2017 - Sep. 2019

Developed responsive web applications using modern JavaScript frameworks with React and Vue.js. Implemented responsive design principles and optimized web interfaces for multiple devices and browsers. Collaborated with design and backend teams to integrate frontend components and ensure adherence to project specifications.

Frontend Web Developer (DSC-Medien)

Aug. 2016 - Sep. 2017

Created and maintained responsive web solutions using HTML5, CSS3, and JavaScript. Developed custom WordPress themes and plugins to meet client-specific requirements. Implemented search engine and loading performance optimizations.

Trainee (SIXWASNINE)

Aug. 2013 - Jul. 2016

Gained experience in communication and advertising, developing marketing strategies and visual communication materials. Assisted in client presentations, brand identity design, and social media content creation. Developed skills in graphic design, copywriting, and project management within a creative agency environment.

Education

Doctor of Philosophy - Ph.D., Trier University

2023 - 2026

summa cum laude (with highest distinction)

A Critical Examination of Large Language Models as Human Simulacra.

Master of Science - MS, Natural Language Processing (Trier University)

2021 - 2023

Grade: 1.2 (very good)

NLP Methodologies, Deep Learning, Language Model Interpretation. Active member of CoDiPho (student council Computational Linguistics & Digital Humanities).

Bachelor of Arts - BA, Computational Linguistics (University of Düsseldorf)

2017 - 2021

Grade: 1.4 (very good)

Programming, Logical & Mathematical Foundations, Corpus Analysis, Parsing, Machine Learning, Computational Linguistic Downstream Tasks.

Mediengestalter Digital und Print (IHK Mittlerer Niederrhein)

2013 - 2016

Grade: 91/100 (good)

Digital Media Design, Frontend Web Development, Visual Communication.

Teaching

Fundamentals of Modern Language Processing

[Lecture]

Summer Term 20[24|25|26], Trier University

Fundamentals of Contemporary (Computational) Linguistics

[Lecture]

Winter Term 20[24|25], Trier University

Large Language Models as Silicon Samples

[Student Research Group]

Winter term 202[23|24|25], Trier university

Critical Analysis of AI in Social Sciences

[Guest Lecture]

Summer Term 20[24|25], Trier University

Introduction to Python Programming

[Exercise]

Winter Term 2023, Trier University

Awards

HICSS-59 Best Paper Award

2026 Jan.

Hawaii International Conference on System Sciences (HICSS) — Link

Best Master's Degree 2023, in Natural Language Processing Program

2024 Jan.

Fach Computerlinguistik & Digital Humanities

Best Bachelor's Degree 2021, in Computational Linguistics Program

2022 Jan.

Förderkreis der Allgemeinen Linguistik an der Heinrich-Heine-Universität Düsseldorf — Link

Skills

Core

Natural Language Processing, Machine Learning, Language Models, Agentic AI, API Development

Programming & Tools

Python (uv, Pydantic, Pandas, Polars, FastAPI, scikit-learn, Pytorch, Hugging Face ecosystem), Web Technologies, (Typescript, Astro, Svelte), R (Tidyverse, psych), Writing (LaTeX, Typst)

Miscellaneous

Academic Research, University Teaching, Communication & Presentations