Human Pose Estimation Based on ISAR and Deep Learning

Estimating human posture is a key element of behavior analysis and human activity recognition (HAR) in many applications, such as public surveillance and gaming. Existing contactless human pose estimation (HPE) methods are mostly vision-based, which may violate privacy and lose functionality in harsh weather and poor light conditions. On the other hand, while being robust against these limitations, mm-wave radars provide high-resolution range data but suffer from no/poor angular resolution. In this article, we employ mm-wave radar along with the inverse synthetic aperture radar (ISAR) algorithm to obtain a high-resolution radar image of a moving person in both range and cross-range dimensions and use the binarized ISAR image as input to an HPE model. The HPE model is trained using labels generated by a vision-based HPE model (AlphaPose). We show that the proposed pipeline can estimate pose from afar (e.g., 4–12 m) using real-world data. We present the pipeline in a general case of a multiple-input-multiple-output (MIMO) radar; however, it can work using a single-input-single-output (SISO) radar as well, providing an extremely affordable solution for behavior analysis applications.