Affective State Prediction in a Mobile Setting using Wearable Biometric Sensors and Stylus

Jul 2, 2019·
Dr. Rafael Wampfler
Dr. Rafael Wampfler
,
S. Klingler
,
B. Solenthaler
,
V. R. Schinazi
,
M. Gross
Abstract
We investigate affective state prediction during mobile learning tasks using a combination of wearable biometric sensors and stylus interaction data. Students completing pen-based educational tasks on tablets provide rich streams of physiological signals — including heart rate, electrodermal activity, and motion — alongside stylus pressure, velocity, and trajectory features. We develop and evaluate machine learning models that fuse these heterogeneous inputs to predict self-reported affective states including valence, arousal, and engagement. Our results demonstrate that combining biometric signals with stylus dynamics substantially improves prediction accuracy compared to single-modality approaches.
Type
Publication
In Proceedings of the International Conference on Educational Data Mining (EDM), Montréal, Canada
publications
Affective Computing Mobile Sensing Wearables Educational Data Mining Stylus Interaction
Dr. Rafael Wampfler
Authors
Dr. Rafael Wampfler
Senior Researcher & Lecturer

I am a Senior Researcher & Lecturer at the Computer Graphics Laboratory of ETH Zurich, and a Research Consultant at Disney Research. I am leading the Digital Character AI projects at CGL. My research interests include conversational digital characters, affective computing, human-computer interaction, and applied machine learning.

My vision is to create intelligent digital humans that can naturally communicate, understand, and support people across domains such as education and mental health. My research focuses on multimodal artificial intelligence for interactive digital humans, developing models that combine large language models, affective computing, and data-driven animation to create embodied conversational agents endowed with autonomous agency, consistent values, and beliefs.

My work bridges machine learning, human–computer interaction, and computer graphics to enable AI systems such as Digital Einstein and interactive patient avatars for psychotherapy training and health education.