Visual place recognition for autonomous mobile robot navigation using LoFTR and MAGSAC++

Udink Aulia; Iskandar Hasanuddin; Muhammad Dirhamsyah; Nasaruddin Nasaruddin

doi:10.30811/jpl.v22i2.4992

Visual place recognition for autonomous mobile robot navigation using LoFTR and MAGSAC++

Udink Aulia, Iskandar Hasanuddin, Muhammad Dirhamsyah, Nasaruddin Nasaruddin

Abstract

Autonomous mobile robots are defined as robotic entities capable of independent movement and intelligent decision-making, relying on their ability to perceive and analyze their surroundings, including objects in their environment. In Simultaneous Localization and Mapping (SLAM) systems, loop closure is often achieved through visual place recognition techniques, where the system compares the current visual input with previously observed scenes to identify matches. In computer vision applications, Speeded-Up Robust Features (SURF) and Scale-Invariant Feature Transform (SIFT) are popular feature extraction algorithms used for such as key point detection, matching, and image registration tasks. The choice of inlier threshold should be based on the specific characteristics of the application and the nature of the images being processed. It often requires experimentation and tuning to find the optimal balance between robustness and accuracy. It Utilizes the pre-trained Local Feature Transformer (LoFTR) and MAGSAC++ estimator to address these drawbacks by employing the number of inliers to determine the similarity between two images for visual place recognition. Our experiment demonstrates that the number of inliers can determine the similarity of locations between two images. Scale variations and translation in location significantly influence the resulting number of inliers. Comparing images from the same location and from different locations yields varying numbers of inliers. The number of inliers significantly influences the similarity of locations. At the same location, the number of inliers is above 150, while at different locations, the number is below 150.

Keywords

SLAM; LoFTR;inlier;key point;Visual Place Recognition

Full Text:

PDF

References

C. Masone and B. Caputo, “A Survey on Deep Visual Place Recognition,” IEEE Access, vol. 9, pp. 19516–19547, 2021, doi: 10.1109/ACCESS.2021.3054937.

D. Zhou, Y. Luo, Q. Zhang, Y. Xu, D. Chen, and X. Zhang, “A Lightweight Neural Network for Loop Closure Detection in Indoor Visual SLAM,” Int. J. Comput. Intell. Syst., vol. 16, no. 1, pp. 1–11, 2023, doi: 10.1007/s44196-023-00223-8.

C. Theodorou, V. Velisavljevic, V. Dyo, and F. Nonyelu, “Visual SLAM algorithms and their application for AR, mapping, localization and wayfinding,” Array, vol. 15, no. July, p. 100222, 2022, doi: 10.1016/j.array.2022.100222.

L. Chen, S. Jin, and Z. Xia, “Towards a Robust Visual Place Recognition in Large-Scale vSLAM Scenarios Based on a Deep Distance Learning,” Sensors, vol. 21, 2021, doi: 10.3390/s21010310.

S. Lowry et al., “Visual Place Recognition: A Survey,” IEEE Trans. Robot., vol. 32, no. 1, pp. 1–19, 2016, doi: 10.1109/TRO.2015.2496823.

L. G. Camara and L. Přeučil, “Visual Place Recognition by spatial matching of high-level CNN features,” Rob. Auton. Syst., vol. 133, p. 103625, 2020, doi: 10.1016/j.robot.2020.103625.

S. K. Sharma, K. Jain, and A. K. Shukla, “A Comparative Analysis of Feature Detectors and Descriptors for Image Stitching,” Appl. Sci., vol. 13, no. 10, 2023, doi: 10.3390/app13106015.

S. A. K. Tareen and Z. Saleem, “A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK,” 2018 Int. Conf. Comput. Math. Eng. Technol. Inven. Innov. Integr. Socioecon. Dev. iCoMET 2018 - Proc., vol. 2018-Janua, pp. 1–10, 2018, doi: 10.1109/ICOMET.2018.8346440.

A. Khaliq, S. Ehsan, Z. Chen, M. Milford, and K. McDonald-Maier, “A Holistic Visual Place Recognition Approach Using Lightweight CNNs for Significant ViewPoint and Appearance Changes,” IEEE Trans. Robot., vol. 36, no. 2, pp. 561–569, 2020, doi: 10.1109/TRO.2019.2956352.

K. Yousif, A. Bab-Hadiashar, and R. Hoseinnezhad, “An Overview to Visual Odometry and Visual SLAM: Applications to Mobile Robotics,” Intell. Ind. Syst., vol. 1, no. 4, pp. 289–311, 2015, doi: 10.1007/s40903-015-0032-7.

M. A. K. Niloy et al., “Critical Design and Control Issues of Indoor Autonomous Mobile Robots: A Review,” IEEE Access, vol. 9, pp. 35338–35370, 2021, doi: 10.1109/ACCESS.2021.3062557.

G. A. Acosta-Amaya, J. M. Cadavid-Jimenez, and J. A. Jimenez-Builes, “Three-Dimensional Location and Mapping Analysis in Mobile Robotics Based on Visual SLAM Methods,” J. Robot., vol. 2023, 2023, doi: 10.1155/2023/6630038.

C. Liu, J. Xu, and F. Wang, “A Review of Keypoints’ Detection and Feature Description in Image Registration,” Sci. Program., vol. 2021, 2021, doi: 10.1155/2021/8509164.

M. Aladem and S. A. Rawashdeh, “Lightweight visual odometry for autonomous mobile robots,” Sensors (Switzerland), vol. 18, no. 9, pp. 1–14, 2018, doi: 10.3390/s18092837.

F. Rubio, F. Valero, and C. Llopis-Albert, “A review of mobile robots: Concepts, methods, theoretical framework, and applications,” Int. J. Adv. Robot. Syst., vol. 16, no. 2, pp. 1–22, 2019, doi: 10.1177/1729881419839596.

J. Ma, X. Jiang, A. Fan, J. Jiang, and J. Yan, “Image Matching from Handcrafted to Deep Features: A Survey,” Int. J. Comput. Vis., vol. 129, no. 1, pp. 23–79, 2021, doi: 10.1007/s11263-020-01359-2.

F. Hidalgo, “Evaluation of Several Feature Detectors / Extractors on Underwater Images towards vSLAM,” pp. 1–16, 2020, doi: 10.3390/s20154343.

P. Adhikari, B. Roy, O. Sinkar, M. Gupta, and C. Ningthoujam, “Experimental Analysis of Feature-Based Image Registration Methods in Combination with Different Outlier Rejection Algorithms for Histopathological Images †,” pp. 1–9, 2023.

M. Wasala, H. Szolc, and T. Kryjak, “An Efficient Real-Time FPGA-Based ORB Feature Extraction for an UHD Video Stream for Embedded Visual SLAM,” 2022.

J. Sun, Z. Shen, Y. Wang, H. Bao, and X. Zhou, “LoFTR: Detector-Free Local Feature Matching with Transformers,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 4, pp. 8918–8927, 2021, doi: 10.1109/CVPR46437.2021.00881.

D. Barath, J. Noskova, M. Ivashechkin, and J. Matas, “MAGSAC++, a fast, reliable and accurate robust estimator,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 1301–1309, 2020, doi: 10.1109/CVPR42600.2020.00138.

A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “ScanNet: Richly-annotated 3D reconstructions of indoor scenes,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 2432–2443, 2017, doi: 10.1109/CVPR.2017.261.

Z. Li and N. Snavely, “MegaDepth: Learning Single-View Depth Prediction from Internet Photos,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 2041–2050, 2018, doi: 10.1109/CVPR.2018.00218.

C. Riu, V. Nozick, and P. Monasse, “Automatic RANSAC by Likelihood Maximization,” Image Process. Line, vol. 12, pp. 27–49, 2022, doi: 10.5201/ipol.2022.357.

D. Barath, J. Noskova, and J. Matas, “Marginalizing Sample Consensus,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 11, pp. 8420–8432, 2022, doi: 10.1109/TPAMI.2021.3103562.

D. Barath, M. Ivashechkin, and J. Matas, “Progressive NAPSAC: sampling from gradually growing neighborhoods,” 2019, [Online]. Available: http://arxiv.org/abs/1906.02295

E. Riba, D. Mishkin, D. Ponsa, E. Rublee, and G. Bradski, “Kornia: An open source differentiable computer vision library for PyTorch,” Proc. - 2020 IEEE Winter Conf. Appl. Comput. Vision, WACV 2020, pp. 3663–3672, 2020, doi: 10.1109/WACV45572.2020.9093363.

DOI: http://dx.doi.org/10.30811/jpl.v22i2.4992