[ Skip to the content ]

Institute of Formal and Applied Linguistics

at Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic


[ Back to the navigation ]

Publication


Year 2016
Type in proceedings
Status in press
Language English
Author(s) Birch, Alexandra Abend, Omri Bojar, Ondřej Haddow, Barry
Title HUME: Human UCCA-Based Evaluation of Machine Translation
English title HUME: Human UCCA-Based Evaluation of Machine Translation
Czech title HUME: Ruční hodnocení kvality překladu založené na sémantické anotaci UCCA
Proceedings 2016: Stroudsburg, PA, USA: EMNLP 2016: Proceedings of the Conference on Empirical Methods in Natural Language Processing EMNLP 2016
How published online
URL https://arxiv.org/abs/1607.00030
Supported by 2015-2018 H2020-ICT-2014-1-644402 (Himl (Health in my Language))
Czech abstract Článek popisuje novou metodu ručního hodnocení kvality strojového překladu založenou na sémantické anotaci vstupní věty.
English abstract Human evaluation of machine translation normally uses sentence-level measures such as relative ranking or adequacy scales. However, these provide no insight into possible errors, and do not scale well with sentence length. We argue for a semantics-based evaluation, which captures what meaning components are retained in the MT output, thus providing a more fine-grained analysis of translation quality, and enabling the construction and tuning of semantics-based MT.We present a novel human semantic evaluation measure, Human UCCA-based MT Evaluation (HUME), building on the UCCA semantic representation scheme. HUME covers a wider range of semantic phenomena than previous methods and does not rely on semantic annotation of the potentially garbled MT output. We experiment with four language pairs, demonstrating HUME’s broad applicability, and report good inter-annotator agreement rates and correlation with human adequacy scores.
Specialization linguistics ("jazykověda")
Confidentiality default – not confidential
Open access no
Address* Stroudsburg, PA, USA
Month* November
Publisher* Association for Computational Linguistics
Creator: Common Account
Created: 10/24/16 1:34 AM
Modifier: Common Account
Modified: 10/24/16 1:41 AM
***

Content, Design & Functionality: ÚFAL, 2006–2016. Page generated: Mon Sep 25 13:28:19 CEST 2017

[ Back to the navigation ] [ Back to the content ]

100% OpenAIRE compliant