Affective State Prediction Based on Semi-Supervised Learning from Smartphone Touch Data

Apr 25, 2020·
Dr. Rafael Wampfler
Dr. Rafael Wampfler
,
S. Klingler
,
B. Solenthaler
,
V. R. Schinazi
,
M. Gross
Abstract
Collecting labeled affective state data from naturalistic smartphone use is expensive and time-consuming, limiting the scale of supervised training datasets. We investigate semi-supervised learning as a strategy for improving affective state prediction from smartphone touch data when labeled data is scarce. By leveraging the large quantities of unlabeled touch data naturally generated during everyday device use, our semi-supervised approach learns better feature representations that generalize across users and contexts. We conduct a longitudinal study with ecological momentary assessments and demonstrate that semi-supervised models substantially close the gap with fully supervised counterparts while requiring far fewer labels.
Type
Publication
In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Virtual
publications
Affective Computing Semi-Supervised Learning Mobile Sensing Smartphone Touch Emotion Recognition
Dr. Rafael Wampfler
Authors
Dr. Rafael Wampfler
Senior Researcher & Lecturer

I am a Senior Researcher & Lecturer at the Computer Graphics Laboratory of ETH Zurich, and a Research Consultant at Disney Research. I am leading the Digital Character AI projects at CGL. My research interests include conversational digital characters, affective computing, human-computer interaction, and applied machine learning.

My vision is to create intelligent digital humans that can naturally communicate, understand, and support people across domains such as education and mental health. My research focuses on multimodal artificial intelligence for interactive digital humans, developing models that combine large language models, affective computing, and data-driven animation to create embodied conversational agents endowed with autonomous agency, consistent values, and beliefs.

My work bridges machine learning, human–computer interaction, and computer graphics to enable AI systems such as Digital Einstein and interactive patient avatars for psychotherapy training and health education.