RehaBot
Overview
RehaBot is an embodied conversational agent designed to support patients in rehabilitation and home-care settings. The avatar represents a medical professional — capable of conducting structured patient interactions, administering assessments, and delivering health education — to help bridge the gap between in-clinic care and independent recovery at home.
This project is developed in collaboration with Inselspital Bern (University Hospital) and Bern University of Applied Sciences (BFH).
Motivation
Rehabilitation after medical treatment — whether from stroke, orthopedic surgery, cardiac events, or chronic disease — requires sustained patient engagement over weeks or months. Yet contact with healthcare professionals is necessarily episodic, leaving long gaps during which patients must self-manage. Lack of guidance, motivation, and timely feedback during these intervals is a major driver of poor rehabilitation outcomes and preventable hospital readmissions.
An embodied conversational agent that patients can interact with at home — to receive reminders, answer questions, conduct structured assessments, and provide health education — addresses this gap directly. By combining medical knowledge with empathetic communication and a human-like embodied presence, RehaBot aims to make professional-quality support continuously available between clinical appointments.
Approach
RehaBot integrates several complementary AI capabilities within a unified embodied avatar system:
- Medical knowledge integration: Structured clinical knowledge relevant to the patient’s rehabilitation pathway, enabling accurate and safe responses to health questions
- Conversational assessment: The ability to administer structured health questionnaires and functional assessments through natural spoken dialogue, adapting pacing and clarification to individual patient needs
- Empathetic communication: Affective modeling that allows the agent to detect and respond to emotional signals in patient speech — frustration, discouragement, anxiety — with appropriate supportive responses
- Health education: Accessible explanations of rehabilitation exercises, medication adherence, warning signs, and self-management strategies, adapted to the patient’s comprehension level
- Patient-professional interface: Structured summaries of patient interactions accessible to supervising clinicians, supporting continuity of care and early detection of clinical deterioration
The system is built on the same core platform as the Digital Einstein project, enabling rapid deployment of new capabilities while maintaining consistent embodied presentation quality.
Key Results
Recent work exploring embodied conversational interfaces for personal health data reflection demonstrates that users who engage with health information through a conversational agent formulate significantly more specific and actionable health plans compared to traditional dashboard-based exploration. Embodied conversation lowers the cognitive burden of interpreting health data and supports a shift from passive data inspection to active health sensemaking.
Research Partners
- Inselspital Bern (University Hospital of Bern)
- Bern University of Applied Sciences (BFH)
I am a Senior Researcher & Lecturer at the Computer Graphics Laboratory of ETH Zurich, and a Research Consultant at Disney Research. I am leading the Digital Character AI projects at CGL. My research interests include conversational digital characters, affective computing, human-computer interaction, and applied machine learning.
My vision is to create intelligent digital humans that can naturally communicate, understand, and support people across domains such as education and mental health. My research focuses on multimodal artificial intelligence for interactive digital humans, developing models that combine large language models, affective computing, and data-driven animation to create embodied conversational agents endowed with autonomous agency, consistent values, and beliefs.
My work bridges machine learning, human–computer interaction, and computer graphics to enable AI systems such as Digital Einstein and interactive patient avatars for psychotherapy training and health education.