A Joint Personality-Emotion Framework for Personality-Consistent Conversational Agents

Sep 16, 2025·
N. Kovačević
,
C. Holz
,
M. Gross
Dr. Rafael Wampfler
Dr. Rafael Wampfler
Abstract
Maintaining personality consistency in conversational agents requires jointly modeling both stable personality traits and dynamic emotional states, since in humans these two dimensions are deeply intertwined. We present a joint personality-emotion framework that learns unified representations of Big Five personality traits and momentary emotions through contrastive learning, enabling conversational agents to generate responses that are both emotionally appropriate and characterologically consistent. The framework decouples emotion from semantic content in a shared embedding space, allowing the same underlying personality to express itself naturally across diverse emotional registers. We evaluate personality consistency and emotional appropriateness through user studies with embodied conversational agents.
Type
Publication
In Proceedings of the 25th International Conference on Intelligent Virtual Agents (IVA ‘25), Berlin, Germany
publications
Personality Emotion Conversational Agents Contrastive Learning Embodied Agents
Dr. Rafael Wampfler
Authors
Dr. Rafael Wampfler
Senior Researcher & Lecturer

I am a Senior Researcher & Lecturer at the Computer Graphics Laboratory of ETH Zurich, and a Research Consultant at Disney Research. I am leading the Digital Character AI projects at CGL. My research interests include conversational digital characters, affective computing, human-computer interaction, and applied machine learning.

My vision is to create intelligent digital humans that can naturally communicate, understand, and support people across domains such as education and mental health. My research focuses on multimodal artificial intelligence for interactive digital humans, developing models that combine large language models, affective computing, and data-driven animation to create embodied conversational agents endowed with autonomous agency, consistent values, and beliefs.

My work bridges machine learning, human–computer interaction, and computer graphics to enable AI systems such as Digital Einstein and interactive patient avatars for psychotherapy training and health education.