Multimodal vehicle security system based on internet of things: integration of fingerprint authentication, face recognition, and GPS tracking

Dodik Wahyu Wiratama; Muhammad Iman Nur Hakim; Helmi Wibowo; Arief Novianto

doi:10.30811/jpl.v24i3.9032

Multimodal vehicle security system based on internet of things: integration of fingerprint authentication, face recognition, and GPS tracking

Dodik Wahyu Wiratama, Muhammad Iman Nur Hakim, Helmi Wibowo, Arief Novianto

Abstract

The surge in vehicle theft in Indonesia has exposed the weaknesses of conventional security systems such as mechanical keys and immobilizers. This study aims to develop an IoT-based vehicle security system integrating fingerprint authentication, facial recognition, and GPS tracking, and evaluates its performance quantitatively. The system was tested on a Toyota Avanza with six users. Facial recognition used a MobileNetV2 CNN model trained with 1,200 local images across four classes (registered, unregistered, masked, and sunglasses) using a learning rate of 0.001, batch size of 32, and 50 epochs. Fingerprint authentication employed minutiae extraction with Euclidean distance matching. The system successfully implemented two-factor authentication. Facial recognition achieved an accuracy of 94.2%, with a False Acceptance Rate (FAR) of 2.1% and a False Rejection Rate (FRR) of 3.7%. Fingerprint authentication reached 91.5% accuracy, with FAR of 4.3% and FRR of 4.2% under dry finger conditions, while FRR increased to 18.5% for scratched fingers. The system detected unregistered users and triggered engine shutdown while sending photos and GPS coordinates through Telegram. GPS tracking achieved 99.3% positional accuracy. The results demonstrate the feasibility of multimodal IoT-based vehicle security, although performance remains sensitive to lighting conditions, face coverings, and finger surface conditions.

Keywords

Vehicle Security System; Fingerprint; Face Recognition; GPS; Internet of Things (IoT); Raspberry Pi

Full Text:

PDF

References

Badan Pusat Statistik, “Jumlah Kendaraan Bermotor Menurut Provinsi dan Jenis Kendaraan (unit), 2023,” Badan Pusat Statistik. Accessed: Dec. 04, 2025. [Online]. Available: https://www.bps.go.id/id/statistics-table/3/VjJ3NGRGa3dkRk5MTlU1bVNFOTVVbmQyVURSTVFUMDkjMw==/jumlah-kendaraan-bermotor-menurut-provinsi-dan-jenis-kendaraan--unit---2023.html

J. H. Lee, B. S. Hyeon, O. Y. Jeon, and N. I. Park, “Analysis of Real-Time Operating Systems’ File Systems: Built-in Cameras from Vehicles,” Forensic Science International: Digital Investigation, vol. 44, Mar. 2023, doi: 10.1016/j.fsidi.2023.301500.

M. Waruwu, “Metode Penelitian dan Pengembangan (R&D): Konsep, Jenis, Tahapan dan Kelebihan,” Jurnal Ilmiah Profesi Pendidikan, vol. 9, no. 2, pp. 1220–1230, 2024, doi: 10.29303/jipp.v9i2.2141.

Okpatrioka, “Research And Development (R&D) Penelitian Yang Inovatif,” Pendidikan, Bahasa dan Budaya, vol. 1, no. 1, p. 87, 2023.

I. Maulana, N. Khairunisa, and R. Mufidah, “Deteksi Bentuk Wajah Menggunakan Convolutional Neural Network (CNN),” Jurnal Mahasiswa Teknik Informatika, vol. 7, no. 6, 2023.

I. S. Razaq and B. K. Shukur, “Improved Face Morphing Attack Detection Method Using PCA and Convolutional Neural Network,” Karbala International Journal of Modern Science, vol. 9, no. 2, pp. 316–327, 2023, doi: 10.33640/2405-609X.3298.

R. Aditya and B. Setiaji, “Analysis Of Application Haar Cascade Classifier And Local Binary Pattern Histogram Algorithm In Recognizing Faces With Real-Time Grayscale Images Using Opencv,” vol. 4, no. 1, pp. 179–186, 2023, doi: 10.20884/1.jutif.2022.4.1.491.

A. E. Ibhaze and O. A. Aribeana, “An Electronic and Web-Based Authentication, Identification, and Logging Management System,” Journal of Engineering, vol. 30, no. 01, pp. 1–25, 2024.

A. S. Falohun, T. H. Akin-Olayemi, F. W. Akinleye, O. P. Kehinde, and T. M. Oyelami, “Design and Construction of A Door Securrity Alarm System Based On SMS Verification and Voice Recognition,” International Journal of Advanced Research in Computer Science, vol. 12, no. 3, 2021, doi: 10.26483/ijarcs.v12i3.6705.

J. A. S. Nasution, N. Nurwati, and S. Sudarmin, “Pengaplikasian Finger Print Sebagai Engine Start Pada Kendaraan Bermotor Berbasis Mikrokontroler Arduino Uno,” JUTSI (Jurnal Teknologi dan Sistem Informasi), vol. 1, no. 3, pp. 219–226, 2020.

M. R. Farzan, “Physical distancing detection system using OpenCV based on raspberry pi4,” Journal of Computer Engineering, Electronics and Information Technology, vol. 1, no. 2, pp. 117–128, 2023.

I. Chatisa, Y. A. Syahbana, A. Urip, and A. Wibowo, “A building security monitoring system based on the internet of things (IoT) with illumination-invariant face recognition for object detection,” Kinetik, vol. 4, no. 1, pp. 485–497, 2023.

C. W. Wiguna, J. D. Irawan, and M. Orisa, “Penerapan Metode Convolutional Neural Network Pada Aplikasi Deteksi Wajah Buronan Berbasis WEB,” Jurnal Mahasiswa Teknik Informatika, vol. 6, no. 2, Sep. 2022.

B. Artono, T. Lestariningsih, R. G. P. Yudha, and A. A. Bachri, “Motorcycle Security System Using SMS Warning and GPS Tracking,” Journal of Robotics and Control (JRC), vol. 1, no. 5, pp. 150–155, Sep. 2020, doi: 10.18196/jrc.1531.

I. Kapoor et al., “A nationwide survey on the practice of end-of-life care issues in critical care units in India,” Indian J. Crit. Care Med., vol. 27, no. 5, p. 305, 2023.

A. Bukola, “Development of an Anti-Theft Vehicle Security System using GPS and GSM Technology with Biometric Authentication,” 2020. [Online]. Available: www.ijisrt.com

Y. Sabri, A. Siham, and A. Maizate, “Internet of Things (IoT) based Smart Vehicle Security and Safety System,” 2021. [Online]. Available: www.ijacsa.thesai.org

N. S. Irjanto and N. Surantha, “Home security system with face recognition based on convolutional neural network,” International Journal of Advanced Computer Science and Applications, vol. 11, no. 11, 2020.

E. E. Setiawan and M. Hatta, “Sistem Pengenalan Wajah secara Realtime Menggunakan Metode Viola-Jones dan Principal Component Analysis,” Teknika : Engineering and Sains Journal, vol. 5, no. 1, pp. 15–22, 2021.

N. I. Akanda, M. A. Hossain, M. M. I. Fahad, M. N. Rahman, and Khairunnaher, “Cost-effective and user-friendly vehicle tracking system using GPS and GSM technology based on IoT,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 28, no. 3, pp. 1826–1833, 2022, doi: 10.11591/ijeecs.v28.i3.pp1826-1833.

DOI: http://dx.doi.org/10.30811/jpl.v24i3.9032