Mixed reality projection mapping using Kinect

During surgery tumor tissue is often difficult to identify. This may result in either too close tumor resections with a positive resection margin or too wide resections with increased morbidity or poor cosmetic outcome.  Within this research line we aim to tackle the described shortcomings in cancer surgery by the application of image guidance using projection mapping techniques.

Recently we have built up a setup the back projection of analyzed Hyperspectral (HSI) images of the resected specimen to demonstrate the positive resection margin as real-time feedback during the surgery.

The setup is made of a Kinect v2 sensor for the perception of 3D depth and a projector for projection mapping of the analyzed data back to the specimen.   The developed software in our group is capable of  3D shape reconstruction and also camera-projector calibration.

In the current phase of the project, we are moving from a static situation to a more dynamic situation.  The main goal is to be able, in real-time,  back-project the position of relevant of structures (vessels and tumors)  or analyze HSI images (tumor tissue) on the organ taking into account the deformations and tissue movement.

The main challenges of this project are visual tracking, 3D motion compensation and deformation estimation.

Prerequisites

• Enthusiastic Master student in electrical engineering, biomedical engineering, computer science, technical medicine or a related field
• A good team player with excellent communication skills
• A creative solution-finder

Duration: 10 weeks (M2) or 40 weeks (M3)

Start date: a.s.a.p.