On-Device Execution of Deep Learning Models on HoloLens2for Real-Time Augmented Reality Medical Applications

Onderzoeksoutput: Articlepeer review

2 Citaten (Scopus)
54 Downloads (Pure)

Samenvatting

The integration of Deep Learning (DL) models with the HoloLens2 Augmented Reality (AR) headset has enormous potential for real-time AR medical applications. Currently, most applications execute the models on an external server that communicates with the headset via Wi-Fi. This client-server architecture introduces undesirable delays and lacks reliability for real-time applications. However, due to HoloLens2's limited computation capabilities, running the DL model directly on the device and achieving real-time performances is not trivial. Therefore, this study has two primary objectives: (i) to systematically evaluate two popular frameworks to execute DL models on HoloLens2-Unity Barracuda and Windows Machine Learning (WinML)-using the inference time as the primary evaluation metric; (ii) to provide benchmark values for state-of-the-art DL models that can be integrated in different medical applications (e.g., Yolo and Unet models). In this study, we executed DL models with various complexities and analyzed inference times ranging from a few milliseconds to seconds. Our results show that Unity Barracuda is significantly faster than WinML (p-value < 0.005). With our findings, we sought to provide practical guidance and reference values for future studies aiming to develop single, portable AR systems for real-time medical assistance.

Originele taal-2English
Aantal pagina's15
TijdschriftSensors
Volume23
Nummer van het tijdschrift21
DOI's
StatusPublished - 25 okt 2023

Vingerafdruk

Duik in de onderzoeksthema's van 'On-Device Execution of Deep Learning Models on HoloLens2for Real-Time Augmented Reality Medical Applications'. Samen vormen ze een unieke vingerafdruk.

Citeer dit