Immersive Conversations with Digital Einstein: Linking a Physical System and AI

Dec 3, 2024·
Dr. Rafael Wampfler
Dr. Rafael Wampfler
,
N. Kovačević
,
P. Witzig
,
C. Yang
,
M. Gross
Abstract
We present an immersive conversational experience with Digital Einstein, an embodied AI agent that brings the historical figure of Albert Einstein to life. This Emerging Technologies demonstration describes the technical integration of a physical animatronic display system with a real-time AI pipeline encompassing speech recognition, knowledge-grounded dialogue management, personality-consistent response generation, and synchronized facial animation. The paper details the architectural decisions, latency optimizations, and real-world deployment lessons from the SIGGRAPH Asia 2024 installation in Tokyo, Japan, where thousands of visitors engaged in live conversations with the Digital Einstein system.
Type
Publication
In SIGGRAPH Asia 2024 Emerging Technologies (SA ‘24), Tokyo, Japan
publications
Dr. Rafael Wampfler
Authors
Senior Researcher & Lecturer

I am a Senior Researcher & Lecturer at the Computer Graphics Laboratory of ETH Zurich, and a Research Consultant at Disney Research. I am leading the Digital Character AI projects at CGL. My research interests include conversational digital characters, affective computing, human-computer interaction, and applied machine learning.

My vision is to create intelligent digital humans that can naturally communicate, understand, and support people across domains such as education and mental health. My research focuses on multimodal artificial intelligence for interactive digital humans, developing models that combine large language models, affective computing, and data-driven animation to create embodied conversational agents endowed with autonomous agency, consistent values, and beliefs.

My work bridges machine learning, human–computer interaction, and computer graphics to enable AI systems such as Digital Einstein and interactive patient avatars for psychotherapy training and health education.