SegNet4D

Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud

Neng Wang, Ruibin Guo, Chenghao Shi, Ziyue Wang, Hui Zhang Huimin Lu, Zhiqiang Zheng Xieyuanli Chen,
National University of Defense Technology, Changsha, China.
SegNet4D Demo

Abstract

TL;DR: 4D LiDAR semantic segmentation, also referred to as multi-scan semantic segmentation, plays a crucial role in enhancing the environmental understanding capabilities of autonomous robots. It classifies the semantic category of each LiDAR measurement point and detects whether it is dynamic, a critical ability for tasks like obstacle avoidance and autonomous navigation.

Existing approaches often rely on computationally heavy 4D convolutions or recursive networks, which result in poor real-time performance, making them unsuitable for online robotics and autonomous driving applications.

In this paper, we introduce SegNet4D, a novel real-time 4D semantic segmentation network offering both efficiency and strong semantic understanding. SegNet4D addresses 4D segmentation as two tasks: single-scan semantic segmentation and moving object segmentation, each tackled by a separate network head. Both results are combined in a motion-semantic fusion module to achieve comprehensive 4D segmentation. Additionally, instance information is extracted from the current scan and exploited for instance-wise segmentation consistency.

Our approach surpasses state-of-the-art in both multi-scan semantic segmentation and moving object segmentation while offering greater efficiency, enabling real-time operation. Besides, its effectiveness and efficiency have also been validated on a real-world unmanned ground platform.

Video

BibTeX


      @article{wang2024segnet4d,
        title     = {{SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud}},
        author    = {Neng, Wang and Ruibin, Guo and Chenghao, Shi and Ziyue, Wang and Hui, Zhang and Huimin, Lu and Zhiqiang, Zheng and Xieyuanali, Chen},
        journal   = {arXiv},
        year      = {2024}
      }