Multimodal Dialog Act Classification for Conversations With Digital Characters
A multimodal dialog act classifier integrating text and acoustic features for real-time classification in conversations with digital characters. CUI 2024.
p.-witzig
Dr. Rafael Wampfler
Senior Researcher & Lecturer
I am a Senior Researcher & Lecturer at the Computer Graphics Laboratory of ETH Zurich, and a Research Consultant at Disney Research. I am leading the Digital Character AI projects at CGL. My research interests include conversational digital characters, affective computing, human-computer interaction, and applied machine learning.
My vision is to create intelligent digital humans that can naturally communicate, understand, and support people across domains such as education and mental health. My research focuses on multimodal artificial intelligence for interactive digital humans, developing models that combine large language models, affective computing, and data-driven animation to create embodied conversational agents endowed with autonomous agency, consistent values, and beliefs.
My work bridges machine learning, human–computer interaction, and computer graphics to enable AI systems such as Digital Einstein and interactive patient avatars for psychotherapy training and health education.
•
•
1 min read
Chatbots With Attitude: Enhancing Chatbot Interactions Through Dynamic Personality Infusions
Dynamic personality infusion for chatbots — modulating expressed Big Five personality traits at inference time to improve user engagement and interaction quality. CUI 2024.
n.-kovacevic
The Personality Dimensions GPT-3 Expresses During Human-Chatbot Interactions
A large-scale empirical characterization of the personality dimensions GPT-3 expresses during human-chatbot interaction, using Big Five psychometrics. Published in ACM IMWUT 2024.
n.-kovacevic
Personality Trait Recognition Based on Smartphone Typing Characteristics in the Wild
Personality trait recognition from everyday smartphone typing dynamics in naturalistic conditions, using deep learning on keystroke patterns. IEEE Transactions on Affective …
n.-kovacevic
Affective State Prediction from Smartphone Touch and Sensor Data in the Wild
Multimodal affective state prediction from smartphone touch and sensor data in naturalistic conditions, using deep learning fusion. CHI 2022.
Image Reconstruction of Tablet Front Camera Recordings in Educational Settings
Image reconstruction from tablet front camera recordings for engagement analysis in educational settings. EDM 2020.
Glyph-Based Visualization of Affective States
Glyph-based visualization technique for representing multimodal affective state data, designed for intuitive perception and scalable display. EuroVis 2020.
n.-kovacevic
Affective State Prediction Based on Semi-Supervised Learning from Smartphone Touch Data
Semi-supervised learning for affective state prediction from smartphone touch data, leveraging abundant unlabeled naturalistic data. CHI 2020.
Affective State Prediction in a Mobile Setting using Wearable Biometric Sensors and Stylus
Affective state prediction during mobile learning using wearable biometric sensors and stylus interaction data. EDM 2019.
Efficient Feature Embeddings for Student Classification with Variational Auto-encoders
Variational autoencoder-based feature embeddings for student classification in educational data mining settings. EDM 2017.
s.-klingler