Digital Twin
Description
Digital Twin is an interdisciplinary Design Research Project investigating the augmentation of the planning process for neurosurgical interventions. The human brain is anatomically highly complex and still little understood. At the Charité Berlin, an interdisciplinary team is working on new visualization methods to visualize the functional zones of the brain. Image-guided surgical interventions are standardized, but the planning process is solely digital and screen-based. This Design Research Project explores the need of physical representations and surgical-like tools to enhance the Mental Models of surgeon and surgeon team during planning procedure. The functional prototype combines a physical pointing tool with a camera, which recognizes a representation of the skull and overlays the medical 3D data set on a screen (AR).What is the Topic?
In neurosurgery a pressing need to find new solutions for viewing and interacting with 3D data is expressed as the possibilities of surgical practices are determined by the visualization of a pathology, before and during surgery. Within this Design Research the potentials of hybrid forms of interaction with neurosurgical datasets to plan and communicate elaborately a surgical intervention are explored. Spatial and tool centered interactions are being preferably investigated, allowing surgeons to tap into their field of long learned and highly professional motor behavior (embodied cognition). The resulting functional prototypes are meant to introduce a form of trial acting in the planning process, where surgeon and surgeon team can create a shared mental model of individual patients pathologies, risk areas and surgical concept.
Why does it look like this?
The tool setup for planning consists of three parts: 1. a hand-guided pointer, 2. a standardized skull representation, 3. digital materiality. 1. The shape of the hand-guided pointer depends on the handle width according to common sizes for neurosurgical ultrasound cutting tools, a modified webcam is embedded at the front tip. 2. The skull representation is an androgyne generalized human skull. As it must be recognized by the camera to be superimposed with image data the functional prototype, carries so-called object markers. The random pattern of black stripes helps the camera to identify the object. 3. The digital materiality has been iterated in color, texture, surface and opacity, here it became clear that context dependent material properties should be assigned by the specialist them selves.
What is special?
Digital Twin was a close and fruitful cooperation with scientists and surgeons at eye level. Design research and prototyping were made productive in this project on an actual problem in medicine. Compared to the machine-centered approaches of engineers, this research focuses on the surgeon and his/her professional fine motor specialization. The research also translates approaches of new technologies like AR and object recognition into new and application-oriented contexts.
What is new?
The approach of using AR technologies and hybrid tools to strengthen the mental model of the surgeon before intervention to reduce visual noise through superimposing visualizations during the operation is seen as a potential paradigm shift and will be further explored.
Link to Video