EVALUATING NEURAL MACHINE TRANSLATION SYSTEMS FOR ENGLISH–UZBEK RAILWAY ENGINEERING TERMINOLOGY: SEMANTIC ACCURACY, STRUCTURAL CONSISTENCY, AND NATIONAL STANDARD COMPLIANCE (2021–2025)
Keywords:
neural machine translation, railway engineering terminology, Uzbek language, translation quality assessment, technical translation, O‘zbekiston temir yo‘llari.Abstract
This empirical study evaluates Google Translate, DeepL, and Microsoft Translator in translating 59 contemporary English railway engineering terms (2021–2025) into Uzbek. Translations were assessed across three dimensions: semantic accuracy, structural consistency, and compliance with national railway standards of O‘zbekiston temir yo‘llari. The results indicate that Microsoft Translator achieved the highest semantic accuracy (54.12%) and standard compliance (93.22%), while structural consistency remained low across all systems—only 18.6% of terms were translated identically by all three platforms, and 39.0% showed complete inconsistency. These findings have direct implications for railway documentation, safety communication, and international collaboration in Central Asian railway development.
References
1. Alvarez-Vidal, S., & Oliver, A. (2023). Assessing MT with measures of post-editing effort. Ampersand, 10, 100125. https://doi.org/10.1016/j.amper.2023.100125
2. American Railway Engineering and Maintenance-of-Way Association. (2025). AREMA 2025 manual for railway engineering. Lanham, MD: AREMA.
3. Amrhein, C., Moghe, N., Sennrich, R., & Guillou, L. (2024). Machine translation meta-evaluation through challenge sets: ACES and Span-ACES. Computational Linguistics, 51(1), 73–125. https://doi.org/10.1162/coli_a_00530
4. Bekmurodova, S. (2025). Structural characteristics of railway terminology formation: A comparative study of English and Uzbek. Confrencea, 5(1), 229–243.
5. Bekmurodova, S., & Bekmurodov, J. (2025). Benefits of standardizing global railway terminology in Uzbekistan’s railway system. Nordic Press, 8(0008). https://doi.org/10.5281/zenodo.15166668
6. Bozorbekov, A. (2023). Railway transport terminology: From ancient layers to new layers. Journal of Language and Linguistics, 6(4). https://doi.org/10.5281/zenodo.10091854
7. Chopard, D., Corcoran, P., & Spasić, I. (2024). Word sense disambiguation of acronyms in clinical narratives. Frontiers in Digital Health, 6, 1282043. https://doi.org/10.3389/fdgth.2024.1282043
8. European Union Agency for Railways. (2024). ERA railway terminology collection. Retrieved from https://www.era.europa.eu
9. Garg, K. D., Shekhar, S., Kumar, A., Goyal, V., Sharma, B., Chengoden, R., & Srivastava, G. (2022). Framework for handling rare word problems in NMT using multi-word expressions. Applied Sciences, 12(21), 11038. https://doi.org/10.3390/app122111038
10. International Organization for Standardization. (2024). ISO 9879:2024—Railway applications: Rolling stock terminology. Geneva: ISO.
11. Kurbanova, I. S. (2025). Issues of equivalence and linguistic approaches in the translation of railway terms. Ilmiy Anjumanlar.
12. Lee, S., Lee, J., Moon, H., Park, C., Seo, J., Eo, S., Koo, S., & Lim, H. (2023). A survey on evaluation metrics for machine translation. Mathematics, 11(4), 1006. https://doi.org/10.3390/math11041006
13. Liu, Z., Chen, Y., & Zhang, J. (2023). Neural machine translation of electrical engineering based on integrated convolutional neural networks. Electronics, 12(17), 3604. https://doi.org/10.3390/electronics12173604
14. Odermatt, F., Egressy, B., & Wattenhofer, R. (2023). Cascaded beam search: Plug-and-play terminology-forcing for NMT. arXiv. https://doi.org/10.48550/arXiv.2305.14538
15. Oncevay, A., Smiley, C., & Liu, X. (2025). Domain-specific terminology and machine translation for finance in European languages. In Proceedings of NAACL-HLT 2025 (pp. 1–15). https://doi.org/10.18653/v1/2025.naacl-long.140
16. O‘zbekiston temir yo‘llari. (2024). Railway standards and terminology guidelines. Tashkent: Uzbekistan Railways.
17. Shavarani, H. S., & Sarkar, A. (2021). Better NMT by extracting linguistic information from BERT. arXiv. Retrieved from https://arxiv.org/abs/2104.02831
18. Shavkatov, S. S., & Kurbanova, I. S. (2025). Comparative linguistic analysis of railway terms in English and Uzbek. International Journal of Advance Scientific Research, 4(8), 1–9.
19. Ulitkin, I., Filippova, I., Ivanova, N., et al. (2021). Automatic evaluation of the quality of machine translation of a scientific text. E3S Web of Conferences, 284. https://doi.org/10.1051/e3sconf/202128408001
20. Xu, W., & Carpuat, M. (2021). Rule-based morphological inflection improves neural terminology translation. arXiv. Retrieved from https://arxiv.org/abs/2109.04620
21. Yameng, P., Zhongchi, Z., & Lin, J. (2025). Translation quality assessment of mainstream NMT tools on multidimensional quality metrics. In Proceedings of ICAIE 2025. IEEE. https://doi.org/10.1109/ICAIE64856.2025.11158658
22. Zhang, Z., Syed Abdullah, S. N., & Abdullah, M. A. R. (2025). Translation quality in an evolving paradigm: NMT and LLMs in technical domains. International Journal of English Language Education, 13(2). https://doi.org/10.5296/ijele.v13i2.23057
23. Zhuo, T. Y., Xu, Q., He, X., et al. (2023). Rethinking round-trip translation for MT evaluation. In Findings of ACL 2023 (pp. 1–15). https://doi.org/10.18653/v1/2023.findings-acl.22
24. Zouhar, V., Kloudová, V., Popel, M., et al. (2024). Evaluating optimal reference translations. Natural Language Processing. https://doi.org/10.1017/nlp.2024.3
