EMNLP 2025 - Suzhou

Simon Münker

research
published and presented:

Simon Münker

Fingerprinting LLMs through Survey Item Factor Correlation: A Case Study on Humor Style Questionnaire

Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP)

LLMs increasingly engage with psychological instruments, yet how they represent constructs internally remains poorly understood. We introduce a novel approach to “fingerprinting” LLMs through their factor correlation patterns on standardized psychological assessments to deepen the understanding of LLMs constructs representation. Using the Humor Style Questionnaire as a case study, we analyze how six LLMs represent and correlate humor-related constructs to survey participants. Our results show that they exhibit little similarity to human response patterns. In contrast, participants' subsamples demonstrate remarkably high internal consistency. Exploratory graph analysis further confirms that no LLM successfully recovers the four constructs of the Humor Style Questionnaire. These findings suggest that despite advances in natural language capabilities, current LLMs represent psychological constructs in fundamentally different ways than humans, questioning the validity of application as human simulacra.

Drummers and dragon dancers at the EMNLP 2025 social event in Suzhou

Another conference, another excuse to travel across the globe to talk about language models with people who haven’t seen natural sunlight in months. EMNLP 2025 landed in Suzhou, a surprisingly charming mix of high-tech, ancient canals, and existential AI dread.

Bridge with LED panel ceiling displaying animations and advertisements in Suzhou

A massive bridge with the entire ceiling covered in LED panels displaying shifting animations and advertisements. Such modern. It feels like walking through Cyberpunk 2077, but without bugs and meticulously cleaned.

Local religious community performing ritual ceremony in the front of a pagoda

A local religious community walks in slow circles, chanting to their gods. Watching them perform their ritual felt strangely like praying that the reviewers would show mercy on my latest submission. I should find community for that.

Traditional canal boat modernized with LEDs as tourist transportation vehicle

A traditional canal boat reimagined with bright LED strip lights and outboard motors, filled with tourists and floating through the Venice of China bathed in neon glow alongside dozens of fellow tourists.

Wooden plaques with written wishes and spells hanging in a Suzhou alleyway

Squeezed into an alleyway, hundreds of wooden plaques, a collection of written spells and wishes from locals and visitors praying for financial freedom, good health, and intellectual success.