Digital Einstein

A user interacting with Digital Einstein
projects

Overview

Digital Einstein is a flagship embodied conversational agent that brings the historical figure of Albert Einstein to life through real-time multimodal AI interaction. The system combines speech recognition and synthesis, facial animation, gesture control, and a cognitively grounded language understanding pipeline to deliver immersive, personality-consistent conversations.

Digital Einstein serves three interconnected roles:

  1. Research platform — a testbed for studying human–agent interaction in constrained embodied settings, yielding insights on affective computing, personality modeling, dialog act classification, and conversational AI.
  2. Education platform — a live demonstration of conversational AI and multimodal deep learning deployed in university events, science outreach, and public engagement.
  3. Public engagement tool — reaching thousands of visitors globally at scientific conferences, tech summits, museums, and public events, generating sustained international recognition for ETH Zurich.

Motivation

How can AI systems convincingly portray a well-known historical personality — someone whose knowledge, values, and speaking style are culturally established — in real-time dialogue with arbitrary members of the public? This challenge crystallizes core problems in interactive AI: maintaining factual and characterological consistency, adapting dynamically to unpredictable user inputs, and delivering a compelling embodied experience at scale.

Digital Einstein was conceived as both a scientific challenge and a communication vehicle: making abstract advances in AI tangible for general audiences while simultaneously driving rigorous research on the underlying problems.

Approach

The system is built on a full-pipeline architecture described in the SIGGRAPH 2025 paper “A Platform for Interactive AI Character Experiences”. Key components include:

  • Perception layer: Real-time speech recognition via Microsoft Azure Speech Services and multimodal input processing through a webcam-based vision pipeline, including face detection, user characterization, head pose estimation, and re-identification.
  • Cognitive reasoning layer: Knowledge-grounded dialogue management with integrated response generation, powered by GPT-6, featuring dynamic personality infusion that adapts outputs to user-selectable archetypes: Digital Einstein, Rude Bulldozer, Drama Volcano, Zen Master, and Hashtag Prophet.
  • Generative media: On-the-fly image and sound effect generation that visualizes and acoustically underscores the conversation, enriching the dialogue beyond speech.
  • Animation synthesis: Data-driven facial animation synchronized with speech output using NVIDIA Audio2Face, blended with emotion-conditioned expressions, and complemented by a curated library of motion-captured body animations categorized by avatar state, with an LLM-based classifier selecting the most appropriate animation for each response.
  • Embodiment: A stylized Albert Einstein avatar rendered in Unity on a 65-inch display, integrated into a themed early-20th-century physical environment with spatial audio, a hidden microphone, and physical personality sliders built from potentiometers and an Arduino.

The SIGGRAPH Asia 2024 demonstration paper “Immersive Conversations with Digital Einstein: Linking a Physical System and AI” details the physical installation setup, including the integration of an animatronic head with the real-time AI pipeline at the Tokyo venue.

Key Results

Digital Einstein was demonstrated at 24 events (2023–2026):

2026

  • Zurich Film Festival (Zurich, Switzerland)
  • Zug Innovation Day (Zug, Switzerland)
  • ETH Study Information Days (Zurich, Switzerland)
  • Scientifica (Zurich, Switzerland)
  • Inauguration of ETH Zurich Campus Heilbronn (Heilbronn, Germany) — ribbon-cutting
  • wow² – meet ETH (Zurich, Switzerland)
  • After the Algorithm Festival (Zurich, Switzerland)
  • World Economic Forum, ETH Domain event (Davos, Switzerland)

2025

  • Berlin Science Week (Berlin, Germany) — with panel
  • GITEX Global (Dubai, UAE)
  • Zunftbott (ETH Zurich, Switzerland)
  • SIGGRAPH (Vancouver, Canada)
  • Microsoft Initiative to Advance AI Diffusion in Switzerland (Bern, Switzerland)
  • TECH Conference (Heilbronn, Germany)
  • Polymesse (ETH Zurich, Switzerland)

2024

  • SIGGRAPH Asia (Tokyo, Japan) — Emerging Technologies
  • GITEX Global (Dubai, UAE)
  • Digitaltag (Vaduz, Liechtenstein)
  • Swiss Re Resilience Summit (Rüschlikon, Switzerland)
  • Night of the Wissensstadt (Heilbronn, Germany)
  • OpenForum at the World Economic Forum Annual Meeting (Davos, Switzerland)

2023

  • Roche Digital Tag (Rotkreuz, Switzerland)
  • StageOne (TEDx) (Zurich, Switzerland)
  • Swisscom Business Day (Lucerne, Switzerland)

Media: Digital Einstein was featured in the SRF DOK documentary Mein Avatar und Ich (2026) on Swiss national television, with Rafael Wampfler as a featured researcher.

The project has generated sustained international media coverage and public interest, positioning ETH Zurich as a world leader in embodied conversational AI.

Swiss Ambassador to the UAE, Arthur Mattli, interacting with Digital Einstein at GITEX Global in Dubai.

Swiss Ambassador to the UAE, Arthur Mattli, interacting with Digital Einstein at GITEX Global in Dubai.

Learn More

Primary Publications

R. Wampfler, C. Yang, D. Elste, N. Kovačević, P. Witzig and M. Gross (2025). A Platform for Interactive AI Character Experiences. Proceedings of the SIGGRAPH Conference Papers ‘25 (Vancouver, Canada, August 10–14, 2025), pp. 1–11.

R. Wampfler, N. Kovačević, P. Witzig, C. Yang, M. Gross (2024). Immersive Conversations with Digital Einstein: Linking a Physical System and AI. In SIGGRAPH Asia 2024 Emerging Technologies (SA ‘24) (Tokyo, Japan, December 3–6, 2024).

Embodied Conversational Agents Interactive AI Multimodal AI Digital Humans
Dr. Rafael Wampfler
Authors
Dr. Rafael Wampfler
Senior Researcher & Lecturer

I am a Senior Researcher & Lecturer at the Computer Graphics Laboratory of ETH Zurich, and a Research Consultant at Disney Research. I lead the Digital Character AI team at CGL and, as PI of an SNF Ambizione grant, a line of generative models that produce expressive 3D facial animation from speech and text under explicit control of emotion and expression. My research interests include conversational digital characters, generative animation, affective computing, human-computer interaction, and applied machine learning.

My vision is to create intelligent digital humans that can naturally communicate, understand, and support people across domains such as education and mental health. My research focuses on multimodal artificial intelligence for interactive digital humans, developing models that combine large language models, affective computing, and data-driven animation to create embodied conversational agents endowed with autonomous agency, consistent values, and beliefs.

My work bridges machine learning, human–computer interaction, and computer graphics to enable AI systems such as Digital Einstein and interactive patient avatars for psychotherapy training and health education.