Researchers at Microsoft claim to have created the first machine translation system that is able to translate news articles from Chinese to English with the same level of accuracy as a human.
Personnel from the company's Asia and US labs said their system achieved human parity using a commonly used test set of news stories called newstest2017 developed by industry and academic partners.
External bilingual human evaluators compared the results achieved by the Microsoft researchers to two translations produced independently by humans.
One method used by the researchers is known as dual learning. Each time a sentence was translated from Chinese to English, it was sent back through the system to change it from English to Chinese.
{loadposition sam08}Another method is called deliberation networks and is similar to the practice of editing and revising that writers do repeatedly in order to improve their own efforts.
Xuedong Huang, technical fellow in charge of Microsoft’s speech, natural language and machine translation efforts.
Xuedong Huang, a technical fellow in charge of Microsoft’s speech, natural language and machine translation efforts, said: "Hitting human parity in a machine translation task is a dream that all of us have had. We just didn’t realise we’d be able to hit it so soon.”
Arul Menezes, partner research manager of Microsoft’s machine translation team, said the researchers had tried to prove that their systems could achieve the same level of translation using a language pair — Chinese and English — for which there is a lot of data, on a test set that includes the more commonplace vocabulary of general interest news stories.
“Given the best-case situation as far as data and availability of resources goes, we wanted to find out if we could actually match the performance of a professional human translator,” he said.
“Much of our research is really inspired by how we humans do things,” said Tie-Yan Liu, a principal research manager with Microsoft Research Asia in Beijing. Liu leads a team that worked on this project.
Arul Menezes, partner research manager of Microsoft’s machine translation team.
Ming Zhou, assistant managing director of Microsoft Research Asia and head of a natural language processing group that worked on the project, said the group had also developed two new techniques that would improve the accuracy of their translations.
One technique, called joint training, is used to iteratively boost the English-to-Chinese and Chinese-to-English translation systems. It translates new English sentences into Chinese in order to obtain new sentence pairs which are then used to augment the training dataset used for the opposite direction – Chinese to English.
A second technique, agreement regularisation, generates a translation by reading in different direction in order to obtain a consensus translation.
Zhou said he expects these methods and techniques to be useful for improving machine translation in other languages and situations as well. He said they also could be used to make other AI breakthroughs beyond translation.
“This is an area where machine translation research can apply to the whole field of AI research,” he said.
“Machine translation is much more complex than a pure pattern recognition task,” Zhou said. “People can use different words to express the exact same thing, but you cannot necessarily say which one is better.”
Photos: courtesy Microsoft