Supplementary capture using unmanned ground vehicle for 3D reconstruction model improvement

doi:10.3969/j.issn.1674-8484.2024.05.003

Abstract

Abstract:

A 3D model improving method based on supplementary capture by unmanned ground vehicle was proposed to address the issue of damage and holes in 3D reconstruction models generated from images captured solely by unmanned ground vehicle. This method combined model resolution, triangular mesh structure, and manual point selection to extract areas needing improvement, generated 3D bounding boxes and normal vector information, and utilized heuristic methods to generate supplementary viewpoints. The results show that, under this method's optimization, the low-quality areas of the rough 3D model are significantly improved, with an average reduction of 66% in model projection pixel size. Therefore, this method effectively enhances the quality of 3D model reconstruction, providing a reliable solution for large-scale, detailed outdoor 3D reconstruction.

Key words: 3D reconstruction, model improvement, geometric proxy, viewpoint planning, unmanned ground vehicle

CLC Number:

TP23

YANG Donghui, WANG Yuhao. Supplementary capture using unmanned ground vehicle for 3D reconstruction model improvement[J]. Journal of Automotive Safety and Energy, 2024, 15(5): 650-659.

Figures/Tables 19

References 23

[1]	张帆, 黄先锋, 高云龙, 等. 《实景三维中国建设技术大纲(2021版)》解读与思考[J]. 测绘地理信息, 2021, 46(6): 171-174.
	ZHANG Fan, HUANG Xianfeng, GAO Yunlong, et al. Interpretation and reflection on Technology Outline of National 3D Real Scene (2021 Edition)[J]. J Geomatics, 2021, 46(6): 171-174. (in Chinese)
[2]	ZHOU Linglong, WU Guoxin, ZUO Yunbo, et al. A comprehensive review of vision-based 3D reconstruction methods[J]. Sensors, 2024, 24(7): 2314-2349.
[3]	龚元夫, 熊忠招, 陈关州, 等. 实景三维模型缺陷修复关键技术[J]. 科学技术与工程, 2024, 24(19): 8339-8345.
	GONG Yuanfu, XIONG Zhongzhao, CHEN Guanzhou, et al. Key technologies for repairing defect of 3D real scene model[J]. Sci Tech Engi, 2024, 24 (19) : 8339-8345. (in Chinese)
[4]	陈远芳. 基于倾斜摄影测量与BIM技术的室内外一体化三维场景建模及可视化[D]. 抚州: 东华理工大学, 2023.
	CHEN Yuanfang. Modeling and visualization of indoor and outdoor integrated scene based on oblique photogrammetry and BIM technology[D]. Fuzhou: East China University of Technology, 2023. (in Chinese)
[5]	吴晓婧. 三角网格孔洞修补算法研究[D]. 杭州: 浙江工业大学, 2020.
	WU Xiaojing. Research on hole-filling algorithm for triangular mesh models[D]. Hangzhou: Zhejiang University of Technology, 2019. (in Chinese)
[6]	王伟, 樊宏周, 席光. 参数曲面三角网格生成的改进波前法[J]. 西安交通大学学报, 2014, 48(3): 61-67.
	WANG Wei, FAN Hongzhou, XI Guang. A New algorithm for triangular mesh generation by advancing front technique[J]. J Xi'an Jiaotong Univ, 2014, 48(3): 61-67. (in Chinese)
[7]	HUANG Zitian, YU Yikuan, XU Jiawen, et al. PF-Net: Point fractal network for 3D point cloud completion[C/OL]// 2020 IEEE/CVF Conf Comput Visi Patt Recog (CVPR). 2020: 7659-7667. [2023-12-01]. https://openaccess.thecvf.com/content_CVPR_2020/html/Huang_PF-Net_Point_Fractal_Network_for_3D_Point_Cloud_Completion_CVPR_2020_paper.html
[8]	WEN Xin, XIANG Peng, HAN Zhizhong, et al. PMP-Net++: Point cloud completion by transformer-enhanced multi-step point moving paths[J]. IEEE Trans Patt Anal Mach Intel, 2023, 45(1): 852-867.
[9]	Kaba M D, Uzunbas M G, Lim S N. A reinforcement learning approach to the view planning problem[C]// 2017 IEEE Conf Comput Visi Patt Recog (CVPR). Honolulu,2017: 5094-5102.
[10]	Cieslewski T, Kaufmann E, Scaramuzza D. Rapid exploration with multi-rotors: A frontier selection method for high speed flight[C]// 2017 IEEE/RSJ Int'l Conf Intel Robot Syst (IROS). Vancouver, BC, IEEE, 2017: 2135-2142.
[11]	Koch T, Körner M, Fraundorfer F. Automatic and semantically-aware 3D UAV flight planning for image-based 3D reconstruction[J]. Remote Sensing, 2019, 11(13): 1550-1579.
[12]	Obwald S, Bennewitz M, Burgard W, et al. Speeding-up robot exploration by exploiting background information[J]. IEEE Robot Auto Letts, 2016, 1(2): 716-723.
[13]	Mostegel C, Rumpler M, Fraundorfer F, et al. UAV-based autonomous image acquisition with multi-view stereo quality assurance by confidence prediction[C]// 2016 IEEE Conf Comput Visi Patt Recog Workshop (CVPRW). Las Vegas, 2016: 1-10.
[14]	ZHANG Shuhang, LIU Chun, Haala N. Three-dimensional path planning of uavs imaging for photogrammetric reconstruction[C]// ISPRS Annal Photogram, Remo Sens Spat Info Sci, Nice, 2020, 1: 325-331.
[15]	PENG Cheng, Isler V. Adaptive view planning for aerial 3D reconstruction[C]// 2019 Int'l Conf Robot Auto (ICRA). Montreal, 2019: 2981-2987.
[16]	Smith N, Moehrle N, Goesele M, et al. Aerial path planning for urban scene reconstruction: A continuous optimization method and benchmark[J]. ACM Trans Graph, 2018, 37(6): 1-15.
[17]	LIN Liqiang, LIU Yilin, HU Yue, et al. Capturing, reconstructing, and simulating: The urbanscene3D dataset[C]// ECCV 2022. Cham: Springer Nature Switzerland, 2022: 93-103.
[18]	Maboudi M, Homaei M, Song S, et al. A Review on viewpoints and path planning for UAV-based 3-D reconstruction[J]. IEEE J Select Top Appl Earth Obser Remo Sens, 2023, 16: 5026-5048.
[19]	戴维. 三维封闭三角网格模型中的孔洞修补算法研究与实现[D]. 南京: 南京大学, 2018.
	DAI Wei. Research and implementation of hole repair algorithm in 3D closed triangle mesh model[D]. Nanjing: Nanjing University, 2018. (in Chinese)
[20]	Schubert E, Sander J, Ester M, et al. DBSCAN revisited, revisited: Why and how you should (still) use DBSCAN[J]. ACM Trans Data Syst, 2017, 42(3): 1-21.
[21]	徐杰. 基于3D视觉技术的堆叠工件位姿估计研究[D]. 哈尔滨: 哈尔滨工业大学, 2022.
	XU Jie. Research on pose estimation of stacked workpieces based on 3D vision technology[D]. Harbin: Harbin Institute of Technology, 2021. (in Chinese)
[22]	WANG Feng, ZOU Yang, et al.Del Rey Castillo E, Automated UAV path-planning for high-quality photogrammetric 3D bridge reconstruction[J]. Struct Infrast Engi, 2022, 20(10): 1-20.
[23]	ZHOU Xiaohui, XIE Ke, HUANG Kai, et al. Offsite aerial path planning for efficient urban scene reconstruction[J]. ACM Trans Graph, 2020, 39(6): 1-16.

变量名称	说明
scene	进行射线检测的场景
sample	目标样本点
normal	目标样本点法向量
r	决定候选视点范围的半径
viewpoint_height	候选视点据地面高度
z_max	上方障碍检测高度阈值
z_min	下方障碍检测高度阈值
high_resolution_points	OBB中的高分辨率点集合
α	相机长度方向视场角
β	相机宽度方向视场角

变量名称	说明
scene	进行射线检测的场景
sample	目标样本点
normal	目标样本点法向量
r	决定候选视点范围的半径
viewpoint_height	候选视点据地面高度
z_max	上方障碍检测高度阈值
z_min	下方障碍检测高度阈值
high_resolution_points	OBB中的高分辨率点集合
α	相机长度方向视场角
β	相机宽度方向视场角

场景	飞行高度/ cm	覆盖范围/ cm	相机类型	相机焦距/ mm	图像分辨率	图像数量
小镇	7 000	[-10 800, 10 800]*[-10 000, 10 000]	五目相机	20	3840×2160	715
桥	6 000	[-10 800, 10 800]*[-10 000, 10 000]	五目相机	20	3840×2160	715
城堡	5 000	[-10 800, 10 800]*[-10 000, 10 000]	五目相机	20	3840×2160	715

场景	飞行高度/ cm	覆盖范围/ cm	相机类型	相机焦距/ mm	图像分辨率	图像数量
小镇	7 000	[-10 800, 10 800]*[-10 000, 10 000]	五目相机	20	3840×2160	715
桥	6 000	[-10 800, 10 800]*[-10 000, 10 000]	五目相机	20	3840×2160	715
城堡	5 000	[-10 800, 10 800]*[-10 000, 10 000]	五目相机	20	3840×2160	715

场景	视角	优化前后平均投影像素尺寸	优化幅度(优化前后之差占优化前比例) / %
小镇	正面视角	2.74→0.73	73.5
小镇	侧面视角	2.74→0.73	73.5
桥	正面视角	2.02→0.86	57.4
桥	侧面视角	2.02→0.86	57.4
城堡	正面视角	2.22→0.83	62.4
城堡	侧面视角	2.22→0.83	62.4