Research
 

Overview video of selected current research and student projects in the Interactive Graphics and Simulation (IGS) group; prepared November 2018.


Interactive Medical Simulation

Surgical training systems The development of simulations for surgical training, in close collaboration with clinical partners, has been a focus of our group for almost two decades. A key target of the work is to achieve a high level of realism. We strive to go beyond rehearsal of basic manipulative skills, and to enable the training of procedural skills like decision making and problem solving. Furthermore, the integration of the simulation systems into the medical curriculum is tackled. The research activities of the group have also led to the foundation of the spin-off company VirtaMed.

Real-time mesh cutting A central training objective of virtual reality based surgical simulation is the removal of pathologic tissue. This necessitates stable, real-time updates of the underlying mesh representation. We have developed various approaches for cutting into tetrahedral or triangular meshes tailored to specific surgical training system.

Selected movies
Medical Simulation Tissue Cutting Medical Simulation

 

Virtual and Augmented Reality

Colocated visuo-haptic augmentation A key target of recent work has been the extension of the paradigm of augmented reality towards multimodal interaction. Our group has focused on the integration of haptic feedback into visual augmented reality environments. We provided several solutions to allow natural interaction, at the same time both with real and virtual objects. In this context we have developed methods for the precise calibration and integration of haptic interfaces in multimodal AR environments, techniques to improve accuracy and stability of the visual overlay, as well as enhanced visualization methods to improve depth perception.

Haptic augmentation Further, in recent research we also attempt to go beyond mere visual augmentation by providing actual haptic augmentation, i.e. the combination of forces from real and virtual objects. Visuo-haptic augmented reality opens up completely new possibilities of interaction and media presentation, e.g. for education, communication, and collaboration.


Selected movies
Interactive AR Depth in AR Haptic AR

 

Computer Haptics

Data-driven haptic rendering Akin to image-based rendering in computer graphics, we have developed the concept of data-driven haptic rendering. The underlying idea is to acquire interaction data during manipulation of objects, which are subsequently used for data-driven virtual rendering. We have developed a special recording device as well as methods for the integration of different sensor signals for the display. Radial basis functions are used for interpolation of haptic signals acquired from fluids and solids. We also proposed computationally efficient techniques to accelerate the interpolation process. This research domain is a key target of our current work. We strive to build a general framework for multimodal data-driven acquisition and rendering, which allows to visually as well as haptically capture objects during unconstrained interaction for subsequent display.

 

Human Computer Interaction in Medicine

A further direction of the scientific work in our group is the development of new algorithms and systems for medical diagnosis and planning. The leitmotif of this activity is the optimal cooperation between interactive algorithms and the human operator.

Multi-modal data segmentation Extensive research has been invested in recent years into improving interactive segmentation algorithms. However, the human computer interface, a substantial part of an interactive setup, is usually not investigated. The aim of this work is the optimal cooperation between interactive image analysis algorithms and human operators. A visuo-haptic interaction tool for medical segmentation has been designed, which opens the way to virtual endoscopy of the small intestine.

Enhanced surgical planning A further direction is the support of surgical planning procedures with enhanced interfaces. For instance, in joint projects with the University Hospital Zurich and the Balgrist University Hospital, several systems were developed for planning of surgical interventions, on complex fractures of the hip and the shoulder joint, as well as for forearm surgery.