Termal Kamera Görüntülerinin Çoklu Sınıflandırılması için Derin Öğrenme Tabanlı Tekniklerin Performans Karşılaştırılması
Öz
Termal kameralar, cisimlerin sıcaklık farklılıklarını kızılötesi ışın değerlerine bağlı olarak renklendirdiği görüntüleme sistemleridir. Günümüzde başta savunma sanayi olmak üzere sağlık, ziraat, inşaat gibi birçok farklı alanda termal kameralar kullanılmaktadır. Özellikle savunma sanayi alanında kullanılan bu kameralardan elde edilen görüntüler, çeşitli nesnelerin ve canlıların tespiti için büyük önem arz etmektedir. Bu çalışmada termal kamera görüntülerinin sınıflandırması için derin öğrenme tabanlı tekniklerin kapsamlı bir karşılaştırması sunulmaktadır. Çalışmada 7 farklı Evrişimli Sinir Ağları mimarisi ile görüntülerin özellikleri çıkarılmış, 5 farklı sınıflandırma yöntemi ile sınıflandırılması sağlanmıştır. Performans değerlendirmesi için dengesiz çok sınıflı veri kümelerinin sınıflandırılmasına uygun metrikler olan dengeli doğruluk, makro ve mikro ortalama duyarlılık, makro ve mikro ortalama kesinlik, makro ve mikro ortalama F ölçütü metrik değerleri kullanılmıştır. Ayrıca tüm yapıların ayrı ayrı eğitim ve test süreleri karşılaştırılmıştır. Çalışmada en yüksek doğruluk değeri %95.24 ile Resnet101+Softmax ve Resnet50+DVM mimarilerinde elde edilmiştir. Sınıfların eşit ağırlıklı alındığı dengeli doğruluk değerinde ise en yüksek %95,17 ile Resnet101+Softmax mimarisinden elde edilmiştir. Resnet101+Softmax mimarisinde makro ortalamalı kesinlik 0.9579, makro ortalamalı F ölçütü 0.9543 ve mikro ortalamalı F ölçütü 0.9524 değerleri elde edilmiştir. Bu çalışma, küçük ve dengesiz termal görüntüler üzerinde, önceden eğitilmiş ESA ağlarının özellik çıkarımı ile makine öğrenimi sınıflandırıcılarının kullanımının, tamamen eğitilmiş ağlarla elde edilen performansa benzer sonuçlar sağlanabileceğini göstermiştir
Anahtar Kelimeler
Kaynakça
- Aslan S. Mühendislik uygulamalarında termal kamera kullanımı. Yüksek lisans tezi. Hatay: İskenderun Teknik Üniversitesi; 2016.
- Cho Y, Bianchi-Berthouze N, Marquardt N, Julier SJ. Deep thermal imaging. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, New York, NY, USA; 2018. p. 1–13. doi: 10.1145/3173574.3173576.
- Al-Obaidy F, Yazdani F, Mohammadi FA. Fault detection using thermal image based on soft computing methods: Comparative study. Microelectronics Reliability 2017;71:56–64. doi: 10.1016/j.microrel.2017.02.013.
- Sarawade AA, Charniya NN. Detection of faulty integrated circuits in PCB with thermal image processing. In: 2019 International Conference on Nascent Technologies in Engineering (ICNTE), IEEE; 2019. p. 1–6. doi: 10.1109/ICNTE44896.2019.8946061.
- Jiang A, Noguchi R, Ahamed T. Tree trunk recognition in orchard autonomous operations under different light conditions using a thermal camera and Faster R-CNN. Sensors 2022;22(5). doi: 10.3390/s22052065.
- Memari M, Shekaramiz M, Masoum MAS, Seibi AC. Data fusion and ensemble learning for advanced anomaly detection using multi-spectral RGB and thermal imaging of small wind turbine blades. Energies (Basel) 2024;17(3). doi: 10.3390/en17030673.
- Roslidar R, et al. A review on recent progress in thermal imaging and deep learning approaches for breast cancer detection. IEEE Access 2020;8:116176–116194. doi: 10.1109/ACCESS.2020.3004056.
- Bauer J, Mazurkiewicz J. Neural network and optical correlators for infrared imaging based face recognition. In: 5th International Conference on Intelligent Systems Design and Applications (ISDA’05), IEEE; 2005. p. 234–238. doi: 10.1109/ISDA.2005.70.
- Yoshitomi Y, Miyawaki N, Tomita S, Kimura S. Facial expression recognition using thermal image processing and neural network. In: Proceedings 6th IEEE International Workshop on Robot and Human Communication (RO-MAN’97 SENDAI), IEEE; p. 380–385. doi: 10.1109/ROMAN.1997.647016.
- Koukiou G, Anastassopoulos V. Drunk person identification using thermal infrared images. International Journal of Electronic Security and Digital Forensics 2012;4(4). doi: 10.1504/IJESDF.2012.049747.
- Hou F, Zhang Y, Zhou Y, Zhang M, Lv B, Wu J. Review on infrared imaging technology. Sustainability 2022;14(18):11161. doi: 10.3390/su141811161.
- Akula A, Ghosh R, Sardana HK, Predeep P, Thakur M, Varma MKR. Thermal imaging and its application in defence systems. In: AIP Conference Proceedings; 2011. p. 333–335. doi: 10.1063/1.3643540.
- Kaya S, Leloglu UM. Buried and surface mine detection from thermal image time series. IEEE J Sel Top Appl Earth Obs Remote Sens 2017;10(10):4544–4552. doi: 10.1109/JSTARS.2016.2639037.
- Thnh NT, Sahli H, Ho DN. Finite-difference methods and validity of a thermal model for landmine detection with soil property estimation. IEEE Transactions on Geoscience and Remote Sensing 2007;45(3):656–674. doi: 10.1109/TGRS.2006.888862.
- Naga Priya CN, Ashok SD, Maji B, Kumaran KS. Deep learning based thermal image processing approach for detection of buried objects and mines. Engineering Journal 2021;25(3):61–67. doi: 10.4186/ej.2021.25.3.61.
- d’Acremont A, Fablet R, Baussard A, Quin G. CNN-based target recognition and identification for infrared imaging in defense systems. Sensors 2019;19(9):2040. doi: 10.3390/s19092040.
- Canziani A, Paszke A, Culurciello E. An analysis of deep neural network models for practical applications. May 2016.
- Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017;60(6):84–90. doi: 10.1145/3065386.
- Lin M, Chen Q, Yan S. Network in network. Dec. 2013.
- Paszke A, Chaurasia A, Kim S, Culurciello E. ENet: A deep neural network architecture for real-time semantic segmentation. Jun. 2016.
- Szegedy C, et al. Going deeper with convolutions. Sep. 2014.
- Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Sep. 2014.
- He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Dec. 2015.
- Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. Dec. 2015.
- Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, Inception-ResNet and the impact of residual connections on learning. Feb. 2016.
- visible_thermal. Kaggle. Available from: https://www.kaggle.com/datasets/nikhilrajput1112/visible-thermal
- HIT-UAV: A high-altitude infrared thermal dataset. Kaggle. Available from: https://www.kaggle.com/datasets/pandrii000/hituav-a-highaltitude-infrared-thermal-dataset
- Thermal mannequin fall image dataset. Kaggle. Available from: https://www.kaggle.com/datasets/ivannikolov/thermal-mannequin-fall-image-dataset
- Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995;20(3). doi: 10.1023/A:1022627411411.
- Cover TM, Hart PE. Nearest neighbor pattern classification. IEEE Trans Inf Theory 1967;13(1). doi: 10.1109/TIT.1967.1053964.
- Clarke MRB, Duda RO, Hart PE. Pattern classification and scene analysis. J R Stat Soc Ser A 1974;137(3). doi: 10.2307/2344977.
- Fisher RA. The use of multiple measurements in taxonomic problems. Ann Eugen 1936;7(2). doi: 10.1111/j.1469-1809.1936.tb02137.x.
- Huang G, Liu Z, van der Maaten L, Weinberger KQ. Densely connected convolutional networks. Aug. 2016.
- Tharwat A. Classification assessment methods. Applied Computing and Informatics 2021;17(1):168–192. doi: 10.1016/j.aci.2018.08.003.
- Grandini M, Bagli E, Visani G. Metrics for multi-class classification: an overview. Aug. 2020.
- Aslan E. LSTM-ESA hibrit modeli ile MR görüntülerinden beyin tümörünün sınıflandırılması. Adıyaman Üniversitesi Mühendislik Bilimleri Dergisi 2024;11(22):63–81. doi: 10.54365/adyumbd.1391157.
- Takahashi K, Yamamoto K, Kuchiba A, Koyama T. Confidence interval for micro-averaged F1 and macro-averaged F1 scores. Applied Intelligence 2022;52(5):4961–4972. doi: 10.1007/s10489-021-02635-5.
Performance Comparison of Deep Learning Based Techniques for Multiclass Classification of Thermal Camera Images
Öz
Thermal camera systems are imaging systems that color the temperature differences of objects depending on their infrared ray values. Nowadays, thermal cameras are used in many different areas such as defense industry, health, agriculture and construction. Detection of various objects and living creatures in the images taken by these cameras, which are used especially in the field of defense industry, is of great importance. This study presents a comprehensive comparison of deep learning-based techniques for classification of thermal camera images. In the study, the features of the images were extracted with 7 different Convolutional Neural Network architectures and they were classified with 5 different classifiers. For performance evaluation, balanced accuracy, macro and micro average sensitivity, macro and micro average precision, macro and micro average F criterion metric values, which are metrics suitable for classification of unbalanced multi-class data sets, were used. Additionally, the training and testing times of all structures separately were compared. In the study, the highest accuracy value of 95.24% was achieved with ResNet101+Softmax and ResNet50+DVM architectures. The highest balanced accuracy, where class weights were equally considered, was obtained as 95.17% with the ResNet101+Softmax architecture. The ResNet101+Softmax model yielded a macro-averaged precision of 0.9579, a macro-averaged F-measure of 0.9543, and a micro-averaged F-measure of 0.9524. This study demonstrates that, on small and imbalanced thermal images, the utilization of pre-trained ESA networks for feature extraction combined with machine learning classifiers can deliver performance comparable to that achieved with fully trained deep learning models.
Anahtar Kelimeler
Kaynakça
- Aslan S. Mühendislik uygulamalarında termal kamera kullanımı. Yüksek lisans tezi. Hatay: İskenderun Teknik Üniversitesi; 2016.
- Cho Y, Bianchi-Berthouze N, Marquardt N, Julier SJ. Deep thermal imaging. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, New York, NY, USA; 2018. p. 1–13. doi: 10.1145/3173574.3173576.
- Al-Obaidy F, Yazdani F, Mohammadi FA. Fault detection using thermal image based on soft computing methods: Comparative study. Microelectronics Reliability 2017;71:56–64. doi: 10.1016/j.microrel.2017.02.013.
- Sarawade AA, Charniya NN. Detection of faulty integrated circuits in PCB with thermal image processing. In: 2019 International Conference on Nascent Technologies in Engineering (ICNTE), IEEE; 2019. p. 1–6. doi: 10.1109/ICNTE44896.2019.8946061.
- Jiang A, Noguchi R, Ahamed T. Tree trunk recognition in orchard autonomous operations under different light conditions using a thermal camera and Faster R-CNN. Sensors 2022;22(5). doi: 10.3390/s22052065.
- Memari M, Shekaramiz M, Masoum MAS, Seibi AC. Data fusion and ensemble learning for advanced anomaly detection using multi-spectral RGB and thermal imaging of small wind turbine blades. Energies (Basel) 2024;17(3). doi: 10.3390/en17030673.
- Roslidar R, et al. A review on recent progress in thermal imaging and deep learning approaches for breast cancer detection. IEEE Access 2020;8:116176–116194. doi: 10.1109/ACCESS.2020.3004056.
- Bauer J, Mazurkiewicz J. Neural network and optical correlators for infrared imaging based face recognition. In: 5th International Conference on Intelligent Systems Design and Applications (ISDA’05), IEEE; 2005. p. 234–238. doi: 10.1109/ISDA.2005.70.
- Yoshitomi Y, Miyawaki N, Tomita S, Kimura S. Facial expression recognition using thermal image processing and neural network. In: Proceedings 6th IEEE International Workshop on Robot and Human Communication (RO-MAN’97 SENDAI), IEEE; p. 380–385. doi: 10.1109/ROMAN.1997.647016.
- Koukiou G, Anastassopoulos V. Drunk person identification using thermal infrared images. International Journal of Electronic Security and Digital Forensics 2012;4(4). doi: 10.1504/IJESDF.2012.049747.
- Hou F, Zhang Y, Zhou Y, Zhang M, Lv B, Wu J. Review on infrared imaging technology. Sustainability 2022;14(18):11161. doi: 10.3390/su141811161.
- Akula A, Ghosh R, Sardana HK, Predeep P, Thakur M, Varma MKR. Thermal imaging and its application in defence systems. In: AIP Conference Proceedings; 2011. p. 333–335. doi: 10.1063/1.3643540.
- Kaya S, Leloglu UM. Buried and surface mine detection from thermal image time series. IEEE J Sel Top Appl Earth Obs Remote Sens 2017;10(10):4544–4552. doi: 10.1109/JSTARS.2016.2639037.
- Thnh NT, Sahli H, Ho DN. Finite-difference methods and validity of a thermal model for landmine detection with soil property estimation. IEEE Transactions on Geoscience and Remote Sensing 2007;45(3):656–674. doi: 10.1109/TGRS.2006.888862.
- Naga Priya CN, Ashok SD, Maji B, Kumaran KS. Deep learning based thermal image processing approach for detection of buried objects and mines. Engineering Journal 2021;25(3):61–67. doi: 10.4186/ej.2021.25.3.61.
- d’Acremont A, Fablet R, Baussard A, Quin G. CNN-based target recognition and identification for infrared imaging in defense systems. Sensors 2019;19(9):2040. doi: 10.3390/s19092040.
- Canziani A, Paszke A, Culurciello E. An analysis of deep neural network models for practical applications. May 2016.
- Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017;60(6):84–90. doi: 10.1145/3065386.
- Lin M, Chen Q, Yan S. Network in network. Dec. 2013.
- Paszke A, Chaurasia A, Kim S, Culurciello E. ENet: A deep neural network architecture for real-time semantic segmentation. Jun. 2016.
- Szegedy C, et al. Going deeper with convolutions. Sep. 2014.
- Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Sep. 2014.
- He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Dec. 2015.
- Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. Dec. 2015.
- Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, Inception-ResNet and the impact of residual connections on learning. Feb. 2016.
- visible_thermal. Kaggle. Available from: https://www.kaggle.com/datasets/nikhilrajput1112/visible-thermal
- HIT-UAV: A high-altitude infrared thermal dataset. Kaggle. Available from: https://www.kaggle.com/datasets/pandrii000/hituav-a-highaltitude-infrared-thermal-dataset
- Thermal mannequin fall image dataset. Kaggle. Available from: https://www.kaggle.com/datasets/ivannikolov/thermal-mannequin-fall-image-dataset
- Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995;20(3). doi: 10.1023/A:1022627411411.
- Cover TM, Hart PE. Nearest neighbor pattern classification. IEEE Trans Inf Theory 1967;13(1). doi: 10.1109/TIT.1967.1053964.
- Clarke MRB, Duda RO, Hart PE. Pattern classification and scene analysis. J R Stat Soc Ser A 1974;137(3). doi: 10.2307/2344977.
- Fisher RA. The use of multiple measurements in taxonomic problems. Ann Eugen 1936;7(2). doi: 10.1111/j.1469-1809.1936.tb02137.x.
- Huang G, Liu Z, van der Maaten L, Weinberger KQ. Densely connected convolutional networks. Aug. 2016.
- Tharwat A. Classification assessment methods. Applied Computing and Informatics 2021;17(1):168–192. doi: 10.1016/j.aci.2018.08.003.
- Grandini M, Bagli E, Visani G. Metrics for multi-class classification: an overview. Aug. 2020.
- Aslan E. LSTM-ESA hibrit modeli ile MR görüntülerinden beyin tümörünün sınıflandırılması. Adıyaman Üniversitesi Mühendislik Bilimleri Dergisi 2024;11(22):63–81. doi: 10.54365/adyumbd.1391157.
- Takahashi K, Yamamoto K, Kuchiba A, Koyama T. Confidence interval for micro-averaged F1 and macro-averaged F1 scores. Applied Intelligence 2022;52(5):4961–4972. doi: 10.1007/s10489-021-02635-5.
Ayrıntılar
| Birincil Dil | Türkçe |
|---|---|
| Konular | Derin Öğrenme, Sınıflandırma algoritmaları |
| Bölüm | Araştırma Makaleleri |
| Yazarlar | |
| Erken Görünüm Tarihi | 26 Nisan 2025 |
| Yayımlanma Tarihi | 30 Nisan 2025 |
| Gönderilme Tarihi | 25 Temmuz 2024 |
| Kabul Tarihi | 8 Mart 2025 |
| Yayımlandığı Sayı | Yıl 2025 Cilt: 12 Sayı: 25 |
