Traffic Sign Image Segmentation using U-Net Architecture Based on Convolutional Neural Network

Mutaqin Akbar; Budi Sulistiyo Jati; Indah Susilawati; Moh Ahsan

doi:10.35877/454RI.jinav4919

Authors

Mutaqin Akbar Universitas Mercu Buana Yogyakarta
Budi Sulistiyo Jati Universitas Mercu Buana Yogyakarta
Indah Susilawati Universitas Mercu Buana Yogyakarta
Moh Ahsan Universitas PGRI Kanjuruhan Malang

DOI:

https://doi.org/10.35877/454RI.jinav4919

Keywords:

Convolution Neural Network, Deep Learning, Segmentation, Traffic Sign, U-Net

Abstract

Accurate pixel-level segmentation of traffic signs in natural scenes is a critical precursor to reliable sign recognition in intelligent transportation systems. This study investigates the effectiveness of the U-Net architecture for semantic segmentation of traffic signs using a dataset comprising 1,750 training subset and 300 testing subset. The model was trained for 20 epochs with the Adam optimizer (learning rate 0.0002, batch size 2), exhibiting stable convergence: training loss decreased from 0.137729 to 0.05989, while the dice coefficient improved from 0.899644 to 0.956839, and binary IoU rose from 0.894176 to 0.94945. Evaluation on a held-out test set of 300 images demonstrated strong generalization, yielding a dice coefficient of 0.930935, binary IoU of 0.907808, precision of 0.90499, recall of 0.981679, and accuracy of 0.953572. The recall–precision asymmetry indicates a mild tendency toward over-segmentation. The consistent covariation between the dice coefficient and binary IoU across both training and testing phases affirms the internal reliability of the evaluation metrics employed in this study. Qualitative analysis revealed high-fidelity segmentation of polygonal signs, with the model faithfully reproducing the sharp vertices and overall silhouette of both the octagonal and the rhomboid sign. In contrast, the circular sign exhibited a localized boundary failure, characterized by conspicuous concave flattening along its upper-left contour that caused the predicted rim to appear clipped relative to the smooth ground-truth circumference. These findings establish U-Net as a robust segmentation foundation for downstream traffic sign recognition, while motivating future work on architectural refinements to improve boundary-level precision for curvilinear signs and the adoption of recall loss functions to further regulate the recall–precision trade-off and mitigate over-segmentation tendencies.

References

Akbar, M. (2021). Indonesian Traffic Sign Dataset [Graphic]. Zenodo. https://doi.org/10.5281/ZENODO.21194057

Akbar, M., Susilawati, I., Jati, B. S., & Alamsyah, N. (2025). Multi-Task Learning for Traffic Sign Recognition using Multi-Scale Convolutional Neural Networks. International Journal of Advances in Data and Information Systems, 6(2), 391–402. https://doi.org/10.59395/ijadis.v6i2.1406

Akbar, M., Witanti, A., & Susilawati, I. (2019). GPU Accelerated Fuzzy C-Means (FCM) Color Image Segmentation. Compiler, 8(2). https://doi.org/10.28989/compiler.v8i2.455

Ali, S. H., Ahmad, A., Ali, M., Khan, A., & Shaukat, N. (2025). Automated MRI Tumor Segmentation using hybrid U-Net with Transformer and Efficient Attention (Version 2). arXiv. https://doi.org/10.48550/ARXIV.2506.15562

Aulia, I., & Akbar, M. (2026). Segmentasi Citra Rambu Lalu Lintas Menggunakan Arsitektur DeepLabV3+ Berbasis Convolutional Neural Network. Jurnal Sistem Informasi, Teknik Informatika dan Teknologi Pendidikan, 6(1), 64–70. https://doi.org/10.55338/justikpen.v6i1.896

Ben-Abbou, T., El Omrani, H., El Fazazy, K., Mahraz, M. A., Tairi, H., & Riffi, J. (2026). R2KAN-U-Net: A Novel Architecture Integrating Kolmogorov–Arnold Networks with Residual U-Net for Robust Traffic Sign Segmentation. Sensors, 26(12), 3797. https://doi.org/10.3390/s26123797

Benfaress, I., Bouhoute, A., & Zinedine, A. (2025). Advancing Traffic Sign Recognition: Explainable Deep CNN for Enhanced Robustness in Adverse Environments. Computers, 14(3), 88. https://doi.org/10.3390/computers14030088

Boly, S. B., & Akbar, M. (2024). Segmentasi Citra Sel Darah Menggunakan Convolutional Neural Network. Jurnal RESTIKOM?: Riset Teknik Informatika Dan Komputer, 6(2), 390–398. https://doi.org/10.52005/restikom.v6i2.336

Brar, K. K., Goyal, B., Dogra, A., Mustafa, M. A., Majumdar, R., Alkhayyat, A., & Kukreja, V. (2025). Image segmentation review: Theoretical background and recent advances. Information Fusion, 114, 102608. https://doi.org/10.1016/j.inffus.2024.102608

Channa, A., Khan, A., Chandio, A. A., Akbar, A., Memon, S., Hussain, A., & Hamza, A. (2026). PEFT-MedSAM: Efficient Fine-Tuning of Medical Foundation Models for Explainable Skin Lesion Segmentation (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2606.18707

Chen, H., Ali, M. A. H., Nukman, Y., Razak, B. A., Turaev, S., Chen, Y., Zhang, S., Huang, Z., Wang, Z., & Abdulghafor, R. (2024). Computational methods for automatic traffic signs recognition in autonomous driving on road: A systematic review. Results in Engineering, 24, 103553. https://doi.org/10.1016/j.rineng.2024.103553

Dogo, E. M., Afolabi, O. J., & Twala, B. (2022). On the Relative Impact of Optimizers on Convolutional Neural Networks with Varying Depth and Width for Image Classification. Applied Sciences, 12(23), 11976. https://doi.org/10.3390/app122311976

Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010). The Pascal Visual Object Classes (VOC) Challenge. International Journal of Computer Vision, 88(2), 303–338. https://doi.org/10.1007/s11263-009-0275-4

Fredj, H. B., Chabbah, A., Baili, J., Faiedh, H., & Souani, C. (2023). An efficient implementation of traffic signs recognition system using CNN. Microprocessors and Microsystems, 98, 104791. https://doi.org/10.1016/j.micpro.2023.104791

Giuliani, D. (2022). Metaheuristic Algorithms Applied to Color Image Segmentation on HSV Space. Journal of Imaging, 8(1), 6. https://doi.org/10.3390/jimaging8010006

Günthner, T., & Proff, H. (2021). On the way to autonomous driving: How age influences the acceptance of driver assistance systems. Transportation Research Part F: Traffic Psychology and Behaviour, 81, 586–607. https://doi.org/10.1016/j.trf.2021.07.006

He, S., Chen, L., Zhang, S., Guo, Z., Sun, P., Liu, H., & Liu, H. (2021). Automatic Recognition of Traffic Signs Based on Visual Inspection. IEEE Access, 9, 43253–43261. https://doi.org/10.1109/ACCESS.2021.3059052

Hu, T., Deng, Y., Deng, Y., & Ge, A. (2021). Fully Convolutional Network Variations and Method on Small Dataset. 2021 IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE), 40–46. https://doi.org/10.1109/ICCECE51280.2021.9342059

Huang, G., Cao, H., Sun, J., Chen, Z., & Zhou, Z. (2025). Improved road traffic sign recognition from feature reconstruction. Scientific Reports, 15(1), 42656. https://doi.org/10.1038/s41598-025-26757-9

Huang, S.-Y., Hsu, W.-L., Hsu, R.-J., & Liu, D.-W. (2022). Fully Convolutional Network for the Semantic Segmentation of Medical Images: A Survey. Diagnostics, 12(11), 2765. https://doi.org/10.3390/diagnostics12112765

Hutchison, D., Kanade, T., Kittler, J., Kleinberg, J. M., Mattern, F., Mitchell, J. C., Naor, M., Nierstrasz, O., Pandu Rangan, C., Steffen, B., Sudan, M., Terzopoulos, D., Tygar, D., Vardi, M. Y., Weikum, G., Scherer, D., Müller, A., & Behnke, S. (2010). Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition. In K. Diamantaras, W. Duch, & L. S. Iliadis (Eds.), Artificial Neural Networks – ICANN 2010 (Vol. 6354, pp. 92–101). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-15825-4_10

Javed, S., Khan, T. M., Qayyum, A., Sowmya, A., & Razzak, I. (2024). Region Guided Attention Network for Retinal Vessel Segmentation (Version 3). arXiv. https://doi.org/10.48550/ARXIV.2407.18970

Li, S., Wang, S., & Wang, P. (2023). A Small Object Detection Algorithm for Traffic Signs Based on Improved YOLOv7. Sensors, 23(16), 7145. https://doi.org/10.3390/s23167145

Li, X., Ma, Z., Wang, R., Sun, Z., Dai, M., Wang, Y., Liu, Z., & Ye, H. (2026). A Survey on Image Segmentation and Super-resolution Reconstruction in Visual Sensor Networks. ACM Computing Surveys, 58(2), 1–29. https://doi.org/10.1145/3757730

Lim, X. R., Lee, C. P., Lim, K. M., Ong, T. S., Alqahtani, A., & Ali, M. (2023). Recent Advances in Traffic Sign Recognition: Approaches and Datasets. Sensors, 23(10), 4674. https://doi.org/10.3390/s23104674

Manzi, P. R. (2026). Volatility Surface Reconstruction using Deep Learning under No-Arbitrage Constraints (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2605.24031

Memon, M. M., Hashmani, M. A., Junejo, A. Z., Rizvi, S. S., & Raza, K. (2022). Unified DeepLabV3+ for Semi-Dark Image Semantic Segmentation. Sensors, 22(14), 5312. https://doi.org/10.3390/s22145312

Milletari, F., Navab, N., & Ahmadi, S.-A. (2016). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. 2016 Fourth International Conference on 3D Vision (3DV), 565–571. https://doi.org/10.1109/3DV.2016.79

Narkhede, M. M., & Chopade, N. B. (2022). Review of Advanced Driver Assistance Systems and Their Applications for Collision Avoidance in Urban Driving Scenario. In R. Misra, R. K. Shyamasundar, A. Chaturvedi, & R. Omer (Eds.), Machine Learning and Big Data Analytics (Proceedings of International Conference on Machine Learning and Big Data Analytics (ICMLBDA) 2021) (Vol. 256, pp. 253–267). Springer International Publishing. https://doi.org/10.1007/978-3-030-82469-3_23

Neha, F., Bhati, D., Shukla, D. K., Dalvi, S. M., Mantzou, N., & Shubbar, S. (2025). An analytics-driven review of U-Net for medical image segmentation. Healthcare Analytics, 8, 100416. https://doi.org/10.1016/j.health.2025.100416

Nwankpa, C., Ijomah, W., Gachagan, A., & Marshall, S. (2018). Activation Functions: Comparison of trends in Practice and Research for Deep Learning (arXiv:1811.03378). arXiv. http://arxiv.org/abs/1811.03378

Park, M., Oh, S., Park, J., Jeong, T., & Yu, S. (2025). ES-UNet: Efficient 3D medical image segmentation with enhanced skip connections in 3D UNet. BMC Medical Imaging, 25(1), 327. https://doi.org/10.1186/s12880-025-01857-0

Priya, B. L., Jayalakshmy, S., Idayachandran, G., & Kumaran, S. (2022). Performance Analysis of Semantic Segmentation using Optimized CNN based SegNet. 2022 International Conference on Smart Technologies and Systems for Next Generation Computing (ICSTSN), 1–5. https://doi.org/10.1109/ICSTSN53084.2022.9761293

Rajamani, K. T., Rani, P., Siebert, H., ElagiriRamalingam, R., & Heinrich, M. P. (2023). Attention-augmented U-Net (AA-U-Net) for semantic segmentation. Signal, Image and Video Processing, 17(4), 981–989. https://doi.org/10.1007/s11760-022-02302-3

Ramyashree, Utsavi, S. R., Raghavendra, S., Anoop, B. N., & Venugopala, P. S. (2026). Comparative performance analysis of U-Net and DeepLabV3+ for semantic segmentation in traffic environments. Scientific Reports, 16(1), 15614. https://doi.org/10.1038/s41598-026-46740-2

Rani, A. R., Anusha, Y., Cherishama, S. K., & Laxmi, S. V. (2024). Traffic sign detection and recognition using deep learning-based approach with haze removal for autonomous vehicle navigation. E-Prime - Advances in Electrical Engineering, Electronics and Energy, 7, 100442. https://doi.org/10.1016/j.prime.2024.100442

Ren, X., Deng, Z., Ye, J., He, J., & Yang, D. (2023). FCN+: Global Receptive Convolution Makes FCN Great Again (Version 2). arXiv. https://doi.org/10.48550/ARXIV.2303.04589

Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. In N. Navab, J. Hornegger, W. M. Wells, & A. F. Frangi (Eds.), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 (Vol. 9351, pp. 234–241). Springer International Publishing. https://doi.org/10.1007/978-3-319-24574-4_28

Sathyanarayanan, S. (2024). Confusion Matrix-Based Performance Evaluation Metrics. African Journal of Biomedical Research, 4023–4031. https://doi.org/10.53555/AJBR.v27i4S.4345

Seghier, M. L. (2024). Image Segmentation Evaluation With the Dice Index: Methodological Issues. International Journal of Imaging Systems and Technology, 34(6), e23203. https://doi.org/10.1002/ima.23203

Seraj, M., Rosales-Castellanos, A., Shalkamy, A., El-Basyouny, K., & Qiu, T. Z. (2021). The Implications of Weather and Reflectivity Variations on Automatic Traffic Sign Recognition Performance. Journal of Advanced Transportation, 2021, 1–15. https://doi.org/10.1155/2021/5513552

Sun, C., Wen, M., Zhang, K., Meng, P., & Cui, R. (2021). Traffic sign detection algorithm based on feature expression enhancement. Multimedia Tools and Applications, 80(25), 33593–33614. https://doi.org/10.1007/s11042-021-11413-x

Suriya Prakash, A., Vigneshwaran, D., Seenivasaga Ayyalu, R., & Jayanthi Sree, S. (2021). Traffic Sign Recognition using Deeplearning for Autonomous Driverless Vehicles. 2021 5th International Conference on Computing Methodologies and Communication (ICCMC), 1569–1572. https://doi.org/10.1109/ICCMC51019.2021.9418437

Tian, J., Mithun, N., Seymour, Z., Chiu, H.-P., & Kira, Z. (2021). Striking the Right Balance: Recall Loss for Semantic Segmentation. https://doi.org/10.48550/ARXIV.2106.14917

Wang, H., Liu, P., Dou, Q., Song, Y., Luo, M., Han, R., & Zhang, B. (2025). Enhanced edge detection via Dual-branch attention fusion with Canny-assisted supervision. The Visual Computer, 41(12), 9765–9780. https://doi.org/10.1007/s00371-025-03998-3

Wang, J., Wan, X., Li, L., & Wang, J. (2021). An Improved DeepLab Model for Clothing Image Segmentation. 2021 IEEE 4th International Conference on Electronics and Communication Engineering (ICECE), 49–54. https://doi.org/10.1109/ICECE54449.2021.9674326

Wang, T., & Wong, H. S. (2023). A Two-Stage Color Image Segmentation Method Based on Saturation-Value Total Variation. Advances in Applied Mathematics and Mechanics, 15(1), 94–117. https://doi.org/10.4208/aamm.OA-2021-0314

Yang, X., & Li, C. (2026). A new edge detection method for noisy image based on discrete fractional wavelet transform and improved Canny algorithm. Expert Systems with Applications, 298, 129668. https://doi.org/10.1016/j.eswa.2025.129668

Yao, Y., Zhang, Y., Liu, Z., & Yuan, H. (2025). A Bridge Crack Segmentation Algorithm Based on Fuzzy C-Means Clustering and Feature Fusion. Sensors, 25(14), 4399. https://doi.org/10.3390/s25144399

Yeung, M., Sala, E., Schönlieb, C.-B., & Rundo, L. (2022). Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation. Computerized Medical Imaging and Graphics, 95, 102026. https://doi.org/10.1016/j.compmedimag.2021.102026

Zhan, S., Yuan, Q., Lei, X., Huang, R., Guo, L., Liu, K., & Chen, R. (2024). BFNet: A full-encoder skip connect way for medical image segmentation. Frontiers in Physiology, 15, 1412985. https://doi.org/10.3389/fphys.2024.1412985

Zhang, C., Lu, W., Wu, J., Ni, C., & Wang, H. (2024). SegNet Network Architecture for Deep Learning Image Segmentation and Its Integrated Applications and Prospects. Academic Journal of Science and Technology, 9(2), 224–229. https://doi.org/10.54097/rfa5x119