Real-time Neural Dense Elevation Mapping for Urban Terrain with Uncertainty Estimations

Hong Kong University of Science and Technology

 

Abstract & Pipeline

Having good knowledge of terrain information is essential for improving the performance of various downstream tasks on complex terrains, especially for the locomotion and navigation of legged robots. We present a novel framework for neural urban terrain reconstruction with uncertainty estimations. It generates dense robot-centric elevation maps online from sparse LiDAR observations. We design a novel pre-processing and point features representation approach that ensures high robustness and computational efficiency when integrating multiple point cloud frames. A generative Bayesian model then recovers the detailed terrain structures while simultaneously providing the pixel-wise reconstruction uncertainty. We evaluate the proposed pipeline through both simulation and real-world experiments. Our approach achieves high-quality terrain reconstruction with real-time performance on a mobile platform, and the uncertainty estimates may further benefit the downstream tasks of legged robots.

The proposed elevation mapping pipeline. It first pre-processes the observed sparse point clouds to update the statistical point features. A generative Bayesian model then encodes the current point features to generate the dense elevation map and the pixel-wise reconstruction uncertainty while simultaneously returning a binary edge map for multitask learning in the training phase. Two discriminators are adopted to guide the learning of elevation map and edge map generation using adversarial loss and feature matching loss.

Section I: Qualitative Results

Section I-A: Visualized results in simulation and on real-world stairs

Visualized examples of the elevation mapping experiments in a simulated urban scenario (the first row, 8m x 8m), simulated hill scenario (the second row, 8m x 8m), and on real-world stairs (the third row, 3.2m x 3.2m using different approaches, where different colors indicate the height values. For urban terrains and real-world stairs, our approach (N.D.E.M.) provides accurate dense elevation maps with high reconstruction quality and can further recover detailed terrain structures by introducing uncertainty estimation and adopting our statistical point features. In the hills scenario, our approach can still recover the terrain in a global view but not the local features. generate smooth and step-like terrains.

Section I-B: Elevation mapping on real-world stairs (elevation layer & uncertainty layer)

Section II: Quantitative Results

Section II-A: Quantitative elevation mapping performence

Table I presents the mapping performance in simulated urban and hill scenarios. Our models achieve higher mapping accuracy in height values and their gradients compared with the traditional approaches and can generate high-quality maps and rational terrain structures. Even for hills that are not contained in our training dataset, our models still present high mapping performance, indicating their strong generalization capabilities. Our full version (N.D.E.M.) achieves better performance in recovering urban terrain features.

Table II shows the terrain-wise results in simulated urban scenarios. Our models perform better than the traditional approaches on all kinds of evaluated terrains. Our full version (N.D.E.M.) achieves higher mapping performance on “stairs” and “obstacles”, while also having satisfactory results on “slopes”.

Table III shows the performance on real-world stairs. Traditional approaches present unsatisfactory results probably because they may require enough observations for satisfactory performance. Our models generate maps with high accuracy and structural similarity. Our full version (N.D.E.M.) achieves the best performance among all the models, presenting the effectiveness of our approach in real-world scenarios.

Table IV presents the mapping performance in simulated urban scenarios, while we additionally test the performance of applying a median filter after the Telea inpainting approach. The median filter generally further improves the results of “E.M.C. Telea”. However, our approach still outperforms “E.M.C. Telea+Median” in our simulated urban scenarios.

We additionally collected 2k samples in simulated urban scenarios and evaluated the mapping performance with and without adding noises on sensory data or odometry. For the results with sensory and odometry noise, we collected 468 samples each having an observation ratio larger than 80% to evaluate the performance with small occlusions. Table V shows that by removing the noise or the occlusions, the mapping performances are generally improved for all the evaluated approaches. The noise level in our simulation setting does not have an apparent influence on the results. For the scenarios with high observation ratios, the performances of all the approaches are apparently improved, and the traditional approaches perform even better than our models, while our models can still have acceptable performances.

Section II-B: Histogram of height error and plot of uncertainty estimations

The percentage (orange bar plot) of the grids with different levels of error and the corresponding estimated values (purple curve). Our approach provides high-quality mapping results that are accurate enough for various downstream tasks. The uncertainty estimations further reflect the confidence of our model on the generated maps.

BibTeX

If you find our work useful in your research, please consider citing:

@article{yang2023ndem,
    author={Yang, Bowen and Zhang, Qingwen and Geng, Ruoyu and Wang, Lujia and Liu, Ming},
    journal={IEEE Robotics and Automation Letters}, 
    title={Real-Time Neural Dense Elevation Mapping for Urban Terrain With Uncertainty Estimations}, 
    year={2023},
    volume={8},
    number={2},
    pages={696-703},
    doi={10.1109/LRA.2022.3230325}
}