KAM RESURSLI TIL JUFTLIGI SHAROITIDA TARJIMANING AVTOMATIK METRIKALARI VA EKSPERT BAHOSI
DOI:
https://doi.org/10.67227/3gcay147Kalit so‘zlar:
mashina tarjimasi, tarjima sifatini baholash, avtomatik metrikalar, MQM, kam resursli til, o‘zbek tili, rus tili, korrelyatsion tahlil.Abstrak
Maqolada o‘zbek va rus tillari juftligi uchun mashina tarjimasi sifatini baholash metrikalarining ishonchliligi o‘rganiladi. Tadqiqot materiali sifatida uchta funksional uslubga mansub 600 ta segmentdan iborat sinov to‘plami shakllantirildi. Yettita avtomatik metrika natijalari ekspertlarning MQM asosidagi baholari bilan taqqoslandi. Tahlil natijasida avtomatik metrikalar va ekspert baholari o‘rtasidagi moslik darajasini aniqlash imkonini beruvchi takrorlanuvchan baholash protokoli taklif etildi.
Yuklashlar
Havolalar
1. Papineni K., Roukos S., Ward T., Zhu W.-J. BLEU: A Method for Automatic Evaluation of Machine Translation // Proceedings of ACL 2002. – Philadelphia, 2002. – P. 311–318. – Source
2. Callison-Burch C., Osborne M., Koehn P. Re-evaluating the Role of BLEU in Machine Translation Research // Proceedings of EACL 2006. – Trento, 2006. – P. 249–256. – Source
3. Post M. A Call for Clarity in Reporting BLEU Scores // Proceedings of WMT 2018. – Brussels, 2018. – P. 186–191. – Source
4. Snover M., Dorr B., Schwartz R., Micciulla L., Makhoul J. A Study of Translation Edit Rate with Targeted Human Annotation // Proceedings of AMTA 2006. – Cambridge, 2006. – P. 223–231. – Source
5. Banerjee S., Lavie A. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments // Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures. – Ann Arbor, 2005. – P. 65–72. – Source
6. Lavie A., Denkowski M. The METEOR Metric for Automatic Evaluation of Machine Translation // Machine Translation. – 2009. – Vol. 23, No. 2–3. – P. 105–115. – DOI: 10.1007/s10590-009-9059-4
7. Popović M. chrF: Character n-gram F-score for Automatic MT Evaluation // Proceedings of WMT 2015. – Lisbon, 2015. – P. 392–395. – Source
8. Popović M. chrF++: Words Helping Character n-grams // Proceedings of WMT 2017. – Copenhagen, 2017. – P. 612–618. – Source
9. Zhang T., Kishore V., Wu F., Weinberger K. Q., Artzi Y. BERTScore: Evaluating Text Generation with BERT // Proceedings of ICLR 2020. – Addis Ababa, 2020. – Source
10. Rei R., Stewart C., Farinha A. C., Lavie A. COMET: A Neural Framework for MT Evaluation // Proceedings of EMNLP 2020. – Online, 2020. – P. 2685–2702. – Source
11. Rei R., Treviso M., Guerreiro N. M. et al. CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task // Proceedings of WMT 2022. – Abu Dhabi, 2022. – P. 634–645. – Source
12. Conneau A., Khandelwal K., Goyal N. et al. Unsupervised Cross-lingual Representation Learning at Scale // Proceedings of ACL 2020. – Online, 2020. – P. 8440–8451. – Source
13. Freitag M., Foster G., Grangier D., Ratnakar V., Tan Q., Macherey W. Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation // Transactions of the Association for Computational Linguistics. – 2021. – Vol. 9. – P. 1460–1474. – Source
14. Freitag M., Mathur N., Lo C. et al. Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent // Proceedings of WMT 2023. – Singapore, 2023. – P. 578–628. – Source
15. Deutsch D., Foster G., Freitag M. Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibration // Proceedings of EMNLP 2023. – Singapore, 2023. – Source
16. Lommel A., Uszkoreit H., Burchardt A. Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics // Tradumàtica. – 2014. – No. 12. – P. 455–463. – DOI: 10.5565/rev/tradumatica.77
17. NLLB Team, Costa-jussà M. R. et al. Scaling Neural Machine Translation to 200 Languages // Nature. – 2024. – Vol. 630. – P. 841–846. – DOI: 10.1038/s41586-024-07335-x
18. Mirzakhalov J., Babu A., Ataman D. et al. A Large-Scale Study of Machine Translation in the Turkic Languages // Proceedings of EMNLP 2021. – Punta Cana, 2021. – P. 5876–5890. – Source
19. Allaberdiev B., Matlatipov G., Kuriyozov E., Rakhmonov Z. Parallel Texts Dataset for Uzbek-Kazakh Machine Translation // Data in Brief. – 2024. – Vol. 53. – Article 110194. – DOI: 10.1016/j.dib.2024.110194
20. Mamasaidov M., Shopulatov A. Open Language Data Initiative: Advancing Low-Resource Machine Translation for Karakalpak // Proceedings of WMT 2024. – Miami, 2024. – P. 606–613. – Source
21. Karyukin V., Rakhimova D., Karibayeva A. et al. The Neural Machine Translation Models for the Low-Resource Kazakh-English Language Pair // PeerJ Computer Science. – 2023. – Vol. 9. – Article e1224. – DOI: 10.7717/peerj-cs.1224
22. Тураева Г. Х. Проблемы машинного перевода при переводе на узбекский язык // Universum: технические науки. – 2020. – № 10 (79). – Источник
23. ISO 17100:2015. Translation Services. Requirements for Translation Services. – Geneva: ISO, 2015. – Source
24. Koo T. K., Li M. Y. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research // Journal of Chiropractic Medicine. – 2016. – Vol. 15, No. 2. – P. 155–163. – DOI: 10.1016/j.jcm.2016.02.012
