Multimodal user interface for a semi-robotic visual assistance system for image guided neurosurgery

Abstract We developed a visual assistance system for image guided neurosurgery, consisting of a stereoscopic digital camera (exoscope) mounted on a semi-robotic manipulator. In order to minimize the operation time, the application-specific multimodal user interface enables hands-free manipulation of the exoscope. The surgeon wears thereby a head-mounted unit with a binocular display, a head tracker, a microphone and earphones. Different modes of view positioning and adjustment can be selected by voice and controlled by head rotation while pressing a miniature confirmation button mounted on a finger ring or suction device. Apart from the development of the mechatronic and software modules of…