Extending the Range of Depth Cameras using Linear Perspective for Mobile Robot Applications

Reliable depth sensing is essential in robotics for both basic and advanced robot operations. Depth cameras capture depth data that can be used by a robot’s vision system, but the quality of the data is limited. Many depth estimation and completion algorithms have been introduced to process camera data and predict depth in a scene, but extending camera range is a little explored problem. This thesis presents a geometry-based method that applies a Manhattan constraint and regresses onto sparse depth input to interpolate and extrapolate lines in the scene in order to extend range. To evaluate the proposed approach, a long-range RGBD dataset with corresponding LiDAR ground truth is presented. Experiments demonstrate that the proposed method successfully interpolates and extrapolates detected 3D lines in Manhattan scenes given sparse depth data within a few centimeters of error, providing depth information in parts of the scene missing input depth data from the sensor. The proposed approach performs comparably to a baseline method in interpolating depth and outperforms it in extrapolation.