Month: May 2018

Top 5 Reasons Why Enterprises Rely on Machine Translation for Global Expansion

Top 5 Reasons Why Enterprises Rely on Machine Translation for Global Expansion

SDL published a whitepaper regarding the reasons behind why enterprises rely on Machine Translation for global expansion. SDL stated the case in point in the introduction, which is language barriers between companies and their global customers stifle economic growth. In fact, forty-nine percent of executives say a language barrier has stood in the way of a major international business deal. Nearly two-thirds (64 percent) of those same executives say language barriers make it difficult to gain a foothold in international markets. Whether inside or outside your company, your global audiences prefer to read in their native languages. It speeds efficiency, increases receptivity and allows for easier processing of concepts. 

SDL stated this point as a solution to the aforementioned challenge:

To break the language barrier and expand your global and multilingual footprint, there are opportunities to leverage both human translation and machine translation.

Then, the paper compared between human translation and MT from the perspective of usage. For human translation, it is the best for content that is legally binding, as well as high value, branded content. However, human translation can be costly, can take weeks (or even months) to complete and can’t address all of the real-time needs of your business to serve multilingual prospects, partners and customers.

And regarding MT, it is fast becoming an essential complement to human translation efforts. It is well suited for use as part of a human translation process, but also solves high-volume and real-time content challenges that human translation cannot on its own, including the five that are the focus of this white paper.

First reason:  Online user activity and multilingual engagement

Whether it’s a web forum, blog, community content, customer review or a Wiki page, your online user-generated content (UGC) is a powerful tool for customer experience and can be a great opportunity to connect customers around your brand and products. These are rarely translated because the ever-fluctuating content requires real-time translation that is not possible with traditional translation options. However, this content is a valuable resource for resolving problems, providing information, building a brand and delivering a positive customer experience.

Machine translation provides a way for companies to quickly and affordably translate user reviews on e-commerce sites, comments on blogs or within online communities or forums, Wiki content and just about any other online UGC that helps provide support or information to your customers and prospects. While the translation isn’t perfect, its quality is sufficient for its primary purpose: information.

Second reason:  Global customer service and customer relationship management

The goal of any customer service department is to help customers find the right answer – and to stay off the phone. Phone support is typically expensive and inefficient for the company and can be frustrating for the customer. Today, customer service departments are working to enhance relationships with customers by offering support over as many self-service channels as possible, including knowledge base articles, email support and real-time chat.

However, due to its dynamic nature, this content often isn’t translated into different languages, making multilingual customer service agents required instead. Because of its real-time capabilities, capacity to handle large volumes of content and ability to lower costs, machine translation is an extremely attractive option for businesses with global customer support organizations.

There are two key online customer support areas that are strong candidates for machine translation:
• Real-time communication
• Knowledge base articles

Third reason:  International employee collaboration

Your employees are sharing information every day: proposals, product specification, designs, documents. In a multinational company, they’re likely native speakers of languages other than the one spoken at headquarters. While these employees may speak your language very
well, they most likely prefer to review complex concepts in their native languages. Reading in their native languages increases their mental
processing speed and allows them to work better and faster.

Human translation isn’t possible in this scenario because of the time-sensitivity inherent to internal collaboration. But internal knowledge sharing doesn’t need the kind of letter perfect translation that public-facing documents often do. For internal content sharing, machine translation can provide an understandable translation that will help employees transcend language barriers. In addition, by granting all employees access to a machine translation solution, they are able to access and quickly translate external information as well without sending it through a lengthy translation process or exposing it outside of your walls.

This level of multilingual information sharing and information access can dramatically improve internal communications and knowledge sharing, increase employee satisfaction and retention and drive innovation among your teams.

Forth reason:  Online security and protection of intellectual property

In an effort to be resourceful, your employees will likely seek out free translation methods like Google Translate or Microsoft Bing. These public, web-based machine translation tools are effective, but they allow your intellectual property to be mined to improve search results or for other needs. There is a simple test to determine if your company’s information is being submitted through public channels for translation: Simply have your IT department audit your firewalls to determine how much traffic is going to the IP addresses of online translation services. Many companies have been surprised by the volume of information going out of their organization this way.

This security hole can be plugged with a secure, enterprise-grade machine translation hosted on-premises or in a private cloud. With this type of solution, you can give employees a secure translation option for translation of documents, websites and more. And, of course, you’ll protect your valuable intellectual property by keeping it in-house, where it belongs.

Fifth reason:  Translation capacity and turnaround time for internal teams or agencies

Machine translation can improve the capacity and productivity of internal translation departments or language service providers (LSPs) by 30 percent or more and greatly reduces the cost of content translaton. Large enterprises that translate massive volumes have seen increases up to 300 percent in translation productivity when machine translation is used to generate the initial translation, which is then edited by skilled translators.

Here’s how it works: instead of starting with a raw document, translators start with a machine translation, which they review in a post-editing process. Translators edit and fine-tune the content for readability, accuracy and cultural sensitivity. By front-loading the process with a high-quality machine translation, translators are still able to provide high-quality content, but in a fraction of the time. 

Reference: https://bit.ly/2wXRQSt

A Gentle Introduction to Neural Machine Translation

A Gentle Introduction to Neural Machine Translation

One of the earliest goals for computers was the automatic translation of text from one language to another.

Automatic or machine translation is perhaps one of the most challenging artificial intelligence tasks given the fluidity of human language. Classically, rule-based systems were used for this task, which were replaced in the 1990s with statistical methods. More recently, deep neural network models achieve state-of-the-art results in a field that is aptly named neural machine translation.

In this post, you will discover the challenge of machine translation and the effectiveness of neural machine translation models.

After reading this post, you will know:

  • Machine translation is challenging given the inherent ambiguity and flexibility of human language.
  • Statistical machine translation replaces classical rule-based systems with models that learn to translate from examples.
  • Neural machine translation models fit a single model rather than a pipeline of fine-tuned models and currently achieve state-of-the-art results.

Let’s get started.

What is Machine Translation?

Machine translation is the task of automatically converting source text in one language to text in another language.

In a machine translation task, the input already consists of a sequence of symbols in some language, and the computer program must convert this into a sequence of symbols in another language.

— Page 98, Deep Learning, 2016.

Given a sequence of text in a source language, there is no one single best translation of that text to another language. This is because of the natural ambiguity and flexibility of human language. This makes the challenge of automatic machine translation difficult, perhaps one of the most difficult in artificial intelligence:

The fact is that accurate translation requires background knowledge in order to resolve ambiguity and establish the content of the sentence.

— Page 21, Artificial Intelligence, A Modern Approach, 3rd Edition, 2009.

Classical machine translation methods often involve rules for converting text in the source language to the target language. The rules are often developed by linguists and may operate at the lexical, syntactic, or semantic level. This focus on rules gives the name to this area of study: Rule-based Machine Translation, or RBMT.

RBMT is characterized with the explicit use and manual creation of linguistically informed rules and representations.

— Page 133, Handbook of Natural Language Processing and Machine Translation, 2011.

The key limitations of the classical machine translation approaches are both the expertise required to develop the rules, and the vast number of rules and exceptions required.

What is Statistical Machine Translation?

Statistical machine translation, or SMT for short, is the use of statistical models that learn to translate text from a source language to a target language gives a large corpus of examples.

This task of using a statistical model can be stated formally as follows:

Given a sentence T in the target language, we seek the sentence S from which the translator produced T. We know that our chance of error is minimized by choosing that sentence S that is most probable given T. Thus, we wish to choose S so as to maximize Pr(S|T).

— A Statistical Approach to Machine Translation, 1990.

This formal specification makes the maximizing of the probability of the output sequence given the input sequence of text explicit. It also makes the notion of there being a suite of candidate translations explicit and the need for a search process or decoder to select the one most likely translation from the model’s output probability distribution.

Given a text in the source language, what is the most probable translation in the target language? […] how should one construct a statistical model that assigns high probabilities to “good” translations and low probabilities to “bad” translations?

— Page xiii, Syntax-based Statistical Machine Translation, 2017.

The approach is data-driven, requiring only a corpus of examples with both source and target language text. This means linguists are not longer required to specify the rules of translation.

This approach does not need a complex ontology of interlingua concepts, nor does it need handcrafted grammars of the source and target languages, nor a hand-labeled treebank. All it needs is data—sample translations from which a translation model can be learned.

— Page 909, Artificial Intelligence, A Modern Approach, 3rd Edition, 2009.

Quickly, the statistical approach to machine translation outperformed the classical rule-based methods to become the de-facto standard set of techniques.

Since the inception of the field at the end of the 1980s, the most popular models for statistical machine translation […] have been sequence-based. In these models, the basic units of translation are words or sequences of words […] These kinds of models are simple and effective, and they work well for man language pairs

— Syntax-based Statistical Machine Translation, 2017.

The most widely used techniques were phrase-based and focus on translating sub-sequences of the source text piecewise.

Statistical Machine Translation (SMT) has been the dominant translation paradigm for decades. Practical implementations of SMT are generally phrase-based systems (PBMT) which translate sequences of words or phrases where the lengths may differ

— Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, 2016.

Although effective, statistical machine translation methods suffered from a narrow focus on the phrases being translated, losing the broader nature of the target text. The hard focus on data-driven approaches also meant that methods may have ignored important syntax distinctions known by linguists. Finally, the statistical approaches required careful tuning of each module in the translation pipeline.

What is Neural Machine Translation?

Neural machine translation, or NMT for short, is the use of neural network models to learn a statistical model for machine translation.

The key benefit to the approach is that a single system can be trained directly on source and target text, no longer requiring the pipeline of specialized systems used in statistical machine learning.

Unlike the traditional phrase-based translation system which consists of many small sub-components that are tuned separately, neural machine translation attempts to build and train a single, large neural network that reads a sentence and outputs a correct translation.

— Neural Machine Translation by Jointly Learning to Align and Translate, 2014.

As such, neural machine translation systems are said to be end-to-end systems as only one model is required for the translation.

The strength of NMT lies in its ability to learn directly, in an end-to-end fashion, the mapping from input text to associated output text.

— Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, 2016.

Encoder-Decoder Model

Multilayer Perceptron neural network models can be used for machine translation, although the models are limited by a fixed-length input sequence where the output must be the same length.

These early models have been greatly improved upon recently through the use of recurrent neural networks organized into an encoder-decoder architecture that allow for variable length input and output sequences.

An encoder neural network reads and encodes a source sentence into a fixed-length vector. A decoder then outputs a translation from the encoded vector. The whole encoder–decoder system, which consists of the encoder and the decoder for a language pair, is jointly trained to maximize the probability of a correct translation given a source sentence.

— Neural Machine Translation by Jointly Learning to Align and Translate, 2014.

Key to the encoder-decoder architecture is the ability of the model to encode the source text into an internal fixed-length representation called the context vector. Interestingly, once encoded, different decoding systems could be used, in principle, to translate the context into different languages.

… one model first reads the input sequence and emits a data structure that summarizes the input sequence. We call this summary the “context” C. […] A second mode, usually an RNN, then reads the context C and generates a sentence in the target language.

— Page 461, Deep Learning, 2016.

Encoder-Decoders with Attention

Although effective, the Encoder-Decoder architecture has problems with long sequences of text to be translated.

The problem stems from the fixed-length internal representation that must be used to decode each word in the output sequence.

The solution is the use of an attention mechanism that allows the model to learn where to place attention on the input sequence as each word of the output sequence is decoded.

Using a fixed-sized representation to capture all the semantic details of a very long sentence […] is very difficult. […] A more efficient approach, however, is to read the whole sentence or paragraph […], then to produce the translated words one at a time, each time focusing on a different part of he input sentence to gather the semantic details required to produce the next output word.

— Page 462, Deep Learning, 2016.

The encoder-decoder recurrent neural network architecture with attention is currently the state-of-the-art on some benchmark problems for machine translation. And this architecture is used in the heart of the Google Neural Machine Translation system, or GNMT, used in their Google Translate service.

… current state-of-the-art machine translation systems are powered by models that employ attention.

— Page 209, Neural Network Methods in Natural Language Processing, 2017.

Although effective, the neural machine translation systems still suffer some issues, such as scaling to larger vocabularies of words and the slow speed of training the models. There are the current areas of focus for large production neural translation systems, such as the Google system.

Three inherent weaknesses of Neural Machine Translation […]: its slower training and inference speed, ineffectiveness in dealing with rare words, and sometimes failure to translate all words in the source sentence.

— Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, 2016.

Reference: https://bit.ly/2Cx8zxI

How to become a localization project manager

How to become a localization project manager

Excerpts from an article with the same title, written by Olga Melnikova in Multilingual Magazine.  Olga Melnikova is a project manager at Moravia and an adjunct professor at the Middlebury Institute of International Studies. She has ten years of experience in the language industry. She holds an MA in translation and localization management and two degrees in language studies.

I decided to talk to people who have been in the industry for a while, who have seen it evolve and know where it’s going. My main question was: what should a person do to start a localization project manager career? I interviewed several experts who shared their vision and perspectives — academics, industry professionals and recruiters. I spoke with Mimi Moore, account manager at Anzu Global, a recruiting company for the localization industry; Tucker Johnson, managing director of Nimdzi Insights; Max Troyer, translation and localization management program coordinator at MIIS, and Jon Ritzdorf, senior solution architect at Moravia and an adjunct professor at the University of Maryland and at MIIS. All of them are industry veterans and have extensive knowledge and understanding of its processes.

Why localization project management?

The first question is: Why localization project management? Why is this considered a move upwards compared to the work of linguists who are the industry lifeblood? According to Renato Beninatto and Tucker Johnson’s The General Theory of the Translation Company, “project
management is the most crucial function of the LSP. Project management has the potential to most powerfully impact an LSP’s ability to add value to the language services value chain.” “Project managers are absolutely the core function in a localization company,” said Johnson. “It is important to keep in mind that language services providers do not sell translation, they sell services. Project managers are responsible for coordinating and managing all of the resources that need to be coordinated in order to deliver to the client: they are managing time, money, people and technology.


Nine times out of ten, Johnson added, the project manager is the face of the company to the client. “Face-to-face contact and building the relationship are extremely important.” This is why The General Theory of the Translation Company regards project management to be one of the core functions of any language service provider (LSP). This in no way undermines the value of all the other industry players, especially
linguists who do the actual translation work. However, the industry cannot do without PMs because “total value is much higher than the original translations. This added value is at the heart of the language services industry.” This is why clients are happy to pay higher prices to work with massive multiple services providers instead of working directly with translators.

Who are they?

The next question is, how have current project managers become project managers? “From the beginning, when the industry started 20 years
ago, there were no specialized training programs for project managers,” Troyer recounted. “So there were two ways. One is you were a translator, but wanted to do something else — become an editor, for example, or start to manage translators. The other route was people working in a business that goes global. So there were two types of people who would become project managers — former translators or people who were assigned localization as a job task.”

According to Ritzdorf, this is still the case in many companies. “I am working with project managers from three prospective clients right now, all of whom do not have a localization degree and are all in localization positions. Did they end up there because they wanted to? Maybe not. They did not end up there because they said ‘Wow, I really want to become a head of localization.’ They just ended up there by accident, like a lot of people do.”

“There are a lot of people who work in a company and who have never heard of localization, but guess what? It is their job now to do localization, and they have to figure it out all by themselves,” Moore confirmed. “When the company decides to go international, they have to find somebody to manage that,” said Ritzdorf.

Regionalization


The first to mention regionalization was Ritzdorf, and then other interviewees confrmed it exists. Ritzdorf lives on the East Coast of the
United States, but comes to the West Coast to teach at MIIS, so he sees the differences. “There are areas where localization is a thing, which means when you walk into a company, they actually know about localization. Since there are enough people who understand what localization is, they want someone with a background in it.” Silicon Valley is a great example, said Ritzdorf. MIIS is close; there is a

localization community that includes organizations like Women in Localization; and there are networking events like IMUG. “People live and
breathe localization. However, there is a totally different culture in other regions, which is very fragmented. There are tons of little companies in other parts of the US, and the situation there is different. If I am a small LSP owner in Wisconsin or Ohio, what are my chances of finding someone with a degree or experience to fill a localization position for a project manager? Extremely low. This is why I may hire a candidate who has an undergraduate degree in French literature, for example. Or in linguistics, languages — at least something.”

The recruiters’ perspective


Nimdzi Insights conducted an interesting study about hiring criteria for localization project manager positions (Figure 1). Some 75 respondents (both LSPs and clients) were asked how important on a scale of 1 to 5 a variety of qualifications are for project management positions. Te responses show a few trends. Top priorities for clients are previous localization experience and a college degree, followed by years of experience and proficiency in more than one language. Top criteria for LSPs are reputation and a college degree, also followed
by experience and proficiency in more than one language.

Moore said that when clients want to hire a localization project manager, the skills they are looking for are familiarity with computer assisted translation (CAT) tools “and an understanding of issues that can arise during localization — like quality issues, for example. Compared to
previous years, more technical skills are required by both clients and vendors: CAT tools, WorldServer, machine translation knowledge, sometimes WordPress or basic engineering. When I started, they were nice-to-haves, but certainly not mandatory.”

Technical skill is not enough, however. “Both hard and soft skills are important. You need hard skills because the industry has become a lot more technical as far as software, tools and automation are concerned. You need soft skills to deal with external and internal stakeholders, and one of the main things is working under pressure because you are juggling so many things.

Moore also mentioned some red flags that would cause Anzu not to hire a candidate. “Sometimes an applicant does not demonstrate good English skills in phone interviews. Having good communication skills is important for a client-facing position. Also, people sometimes exaggerate their skills or experience. Another red flag is if the person has a bad track record (if they change jobs every nine months, for example).” ‘

Anzu often hires for project management contract positions in large companies. “Clients usually come to us when they need a steady stream of contractors (three or six months), then in three or six months there will be other contractors. Te positions are usually project managers or testers. If you already work fulltime, a contract position may not be that attractive. However, if you are a newcomer or have just graduated, and you want to get some experience, then it is a great opportunity. You would spend three, six or 12 months at a company, and it is a very good line on the résumé.”

Do you need a localization degree? 

There is no firm answer to the question of whether or not you need a degree. If you don’t know what you should do, it can certainly help. Troyer discussed how the localization program at MIIS has evolved to ft current real-world pressures. “The program was first started in 2004, and it started small. We were first giving CAT tools, localization project management and software localization courses. This is
the core you need to become a project manager. Ten the program evolved and we introduced the introduction and then advanced levels to many courses. There are currently four or five courses focusing on translation
technology.” Recent additions to the curriculum include advanced JavaScript classes, advanced project management and program management. Natural language processing and computational linguistics will be added down the road. “The industry is driving this move because students will need skills to go in and localize Siri into many languages,” said Troyer.

The program at MIIS is a two-year master’s. It can be reduced to one year for those who already have experience. There are other degrees
available, as well as certification programs offered by institutions such as the University of Washington and The Localization Institute.

Moore said that though a localization degree is not a must, it has a distinct advantage. A lot of students have internships that give them experience. They also know tools, which makes their résumés better fit clients’ job descriptions.

However, both Troyer and Ritzdorf said you don’t necessarily need a degree. “If you have passion for languages and technology, you can get the training on your own,” said Troyer. “Just teach yourself these skills, network on your own and try to break into the industry.”

The future of localization project management

Automation, artificial intelligence and machine learning are affecting all industries, and localization is not an exception. However, all the interviewees forecast that there will be more localization jobs in the future.

According to Johnson, there is high project management turnover on the vendor side because if a person is a good manager, they never stay in this position for more than five years. “After that, they either get a job on the client’s side to make twice as much money and have a much easier job, or their LSP has to promote them to senior positions such as group manager or program director.”

“There is a huge opportunity to stop doing things that are annoying,” said Troyer. “Automation will let professionals work on the human side
of things and let the machines run 
day-to-day tasks. Letting the machine send files back and forth will allow humans to spend more time looking at texts and thinking about what questions a translator can ask. This will give them more time for building a personal relationship with the client. We are taking these innovations into consideration for the curriculum, and I often spend time during classes asking, ‘How can you automate this?’”

Moore stated that “we have seen automation change workflows over the last ten years and reduce the project manager’s workload, with files being automatically moved through each step in the localization process. Also, automation and machine translation go hand-in-hand to make the process faster, more efficient and cost-effective.”

NEURAL MACHINE TRANSLATION: THE RISING STAR

NEURAL MACHINE TRANSLATION: THE RISING STAR

These days, language industry professionals simply can’t escape hearing about neural machine translation (NMT). However, there still isn’t enough information about the practical facts of NMT for translation buyers, language service providers, and translators. People often ask: is NMT intended for me? How will it change my life?

A Short History and Comparison

At the beginning of time – around the 1970s – the story began with rule-based machine translation (RBMT) solutions. The idea was to create grammatical rule sets for source and target languages, where machine translation is a kind of conversion process between the languages based on these rule sets. This concept works well with generic content, but adding new content, new language pairs, and maintaining the rule set is very time-consuming and expensive.

This problem was solved with statistical machine translation (SMT) around the late ‘80s and early ‘90s. SMT systems create statistical models by analyzing aligned source-target language data (training set) and use them to generate the translation. The advantage of SMT is the automatic learning process and the relatively easy adaptation by simply changing or extending the training set. The limitation of SMT is the training set itself: to create a usable engine, a large database of source-target segments is required. Additionally, SMT is not language independent in the sense that it is highly sensitive to the language combination and has a very hard time dealing with grammatically rich languages.

This is where neural machine translation (NMT) begins to shine: it can look at the sentence as a whole and can create associations between the phrases over an even longer distance within the sentence. The result is a convincing fluency and an improved grammatical correctness compared to SMT.

Statistical MT vs Neural MT

Both SMT and NMT are working on a statistical base and are using source-target language segment pairs as a basis. What’s the difference? What we typically call SMT is actually Phrase Based Statistical Machine Translation (PBSMT), meaning SMT is splitting the source segments into phrases. During the training process, SMT creates a translation model and a language model. The translation model stores the different translations of the phrases and the language model stores the probability of the sequence of phrases on the target side. During the translation phase, the decoder chooses the translation that gives the best result based on these two models. On a phrase or expression level, SMT (or PBSMT) is performing well, but language fluency and grammar is not good.

‘Buch’ is aligned with ‘book’ twice and only once with ‘the’ and ‘a’ – the winner is the ‘Buch’-’book’ combination

Neural Machine Translation, on the other hand, is using neural network-based, deep, machine learning technology. Words or even word chunks are transformed into “word vectors”. This means that ‘dog’ is not only representing the characters d, o and g, but it can contain contextual information from the training data. During the training phase, the NMT system tries to set the parameter weights of the neural network based on the reference values (source-target translation). Words appearing in similar context will get similar word vectors. The result is a neural network which can process source segments and transfer them into target segments. During translation, NMT is looking for a complete sentence, not just chunks (phrases). Thanks to the neural approach, it is not translating words, it’s transferring information and context. This is why fluency is much better than in SMT, but terminology accuracy is sometimes not perfect.

Similar words are closer to each other in a vector space

The Hardware

A popular GPU: NVIDIA Tesla

One big difference between SMT and NMT systems is that NMT requires Graphics Processing Units (GPUs), which were originally designed to help computers process graphics. These GPUs can calculate astonishingly fast – the latest cards have about 3,500 cores which can process data simultaneously. In fact, there is a small ongoing hardware revolution and GPU-based computers are the foundation for almost all deep learning and machine learning solutions. One of the great perks of this revolution is that nowadays, NMT is not only available for large enterprises, but also for small and medium-sized companies as well.

The Software

The main element, or ‘kernel’, of any NMT solution is the so-called NMT toolkit. There are a couple of NMT toolkits available, such as Nematus or openNMT, but the landscape is changing fast and more companies and universities are now developing their own toolkits. Since many of these toolkits are open-source solutions and hardware resources have become more affordable, the industry is experiencing an accelerating speed in toolkit R&D and NMT-related solutions.

On the other hand, as important as toolkits are, they are only one small part of a complex system, which contains frontend, backend, pre-processing and post-processing elements, parsers, filters, converters, and so on. These are all factors for anyone to consider before jumping into the development of an individual system. However, it is worth noting that the success of MT is highly community-driven and would not be where it is today without the open source community.

Corpora

A famous bilingual corpus: the Rosetta Stone

And here comes one of the most curious questions: what are the requirements of creating a well-performing NMT engine? Are there different rules compared to SMT systems? There are so many misunderstandings floating around on this topic that I think it’s a perfect opportunity to go into the details a little bit.

The main rules are nearly the same both for SMT and NMT systems. The differences are mainly that an NMT system is less sensitive and performs better in the same circumstances. As I have explained in an earlier blog post about SMT engine quality, the quality of an engine should always be measured in relation to the particular translation project for which you would like to use it.

These are the factors which will eventually influence the performance of an NMT engine:

Volume

Regardless of you may have heard, volume is still very important for NMT engines just like in the SMT world. There is no explicit rule on entry volumes but what we can safely say is that the bare minimum is about 100,000 segment pairs. There are Globalese users who are successfully using engines created based on 150,000 segments, but to be honest, this is more of an exception and requires special circumstances (like the right language combination, see below). The optimum volume starts around 500,000 segment pairs (2 million words).

Quality

The quality of the training set plays an important role (garbage in, garbage out). Don’t add unqualified content to your engine just to increase the overall size of the training set.

Relevance

Applying the right engine to the right project is the first key to success. An engine trained on automotive content will perform well on car manual translation but will give back disappointing results when you try to use it for web content for the food industry.

This raises the question of whether the content (TMs) should be mixed. If you have enough domain-specific content you shouldn’t necessarily add more out-of-domain data to your engine, but if you have an insufficient volume of domain-specific data then adding generic content (e.g. from public sources) may help improve the quality. We always encourage our Globalese users to try different engine combinations with different training sets.

Content type

Content generated by possible non-native speaking users on a chat forum or marketing material requiring transcreation is always a challenge to any MT system. On the other hand, technical documentation with controlled language is a very good candidate for NMT.

Language combination

Unfortunately, language combination still has an impact on quality. The good news is that NMT has now opened up the option of using machine translation for languages like Japanese, Turkish, or Hungarian –  languages which had nearly been excluded from the machine translation club because of poor results provided by SMT. NMT has also helped solve the problem of long distance dependencies for German and the translation output is much smoother for almost all languages. But English combined with Latin languages still provides better results than, for example, English combined with Russian when using similar volumes and training set quality.

Expectations for the future

Neural Machine Translation is a big step ahead in quality, but it still isn’t magic. Nobody should expect that NMT will replace human translators anytime soon. What you CAN expect is that NMT can be a powerful productivity tool in the translation process and open new service options both for translation buyers and language service providers (see post-editing experience).

Training and Translation Time

When we started developing Globalese NMT, one of the most surprising experiences for us was that the training time was far shorter than we had previously anticipated. This is due to the amazingly fast evolution of hardware and software. With Globalese, we currently have an average training time of 50,000 segments per hour – this means that an average engine with 1 million segments can be trained within one day. The situation is even better when looking at translation times: with Globalese, we currently have an average translation time between 100 and 400 segments per minute, depending on the corpus size, segment length in the translation and training content.

Neural MT Post-editing Experience

One of the great changes neural machine translation brings along is that the overall language quality is much better when compared to the SMT world. This does not mean that the translation is always perfect. As stated by one of our testers: if it is right, then it is astonishingly good quality. The ratio of good and poor translation naturally varies depending on the engine, but good engines can provide about 50% (or even higher) of really good translation target text.

Here are some examples showcasing what NMT post-editors can expect:

DE original:

Der Rechnungsführer sorgt für die gebotenen technischen Vorkehrungen zur wirksamen Anwendung des FWS und für dessen Überwachung.

Reference human translation:

The accounting officer shall ensure appropriate technical arrangements for aneffective functioning of the EWS and its monitoring.

Globalese NMT:

The accounting officer shall ensure the necessary technical arrangements for theeffective use of the EWS and for its monitoring.

As you can see, the output is fluent, and the differences are just preferential ones, more or less. This is highlighting another issue: automated quality metrics like BLEU score are not really sufficient to measure the quality. The example above is only a 50% match in the BLEU score, but if we look at the quality, the rating should be much higher.

Let’s look another example:

EN original

The concept of production costs must be understood as being net of any aid but inclusive of a normal level of profit.

Reference human translation:

Die Produktionskosten verstehen sich ohne Beihilfe, aber einschließlich eines normalen Gewinns.

Globalese NMT:

Der Begriff der Produktionskosten bezieht sich auf die Höhe der Beihilfe, aber einschließlich eines normalen Gewinns.

What is interesting here that the first part of the sentence sounds good, but if you look at the content, the translation is not good. This is an example of a fluent output with a bad translation. This is a typical case in the NMT world and it emphasizes the point that post-editors must examine NMT output differently than they did for SMT – in SMT, bad grammar was a clear indicator that the translation must be post-edited.

Post-editors who used to proof and correct SMT output have to change the way they are working and have to be more careful with proofreading, even if the NMT output looks alright at first glance. Also, services related to light post-editing will change – instead of correcting serious grammatical errors without checking the correctness of translation in order to create some readable content, the focus will shift to sorting out serious mistranslations. The funny thing is that one of the main problems in the SMT world was weak fluency and grammar, and now we have good fluency and grammar as an issue in the NMT world…

And finally:

DE original:

Aufgrund des rechtlichen Status der Beteiligten ist ein solcher Vorgang mit einer Beauftragung des liefernden Standorts und einer Berechnung der erbrachten Leistung verbunden.

Reference human translation:

The legal status of the companies involved in these activities means that this process is closely connected with placing orders at the location that is to supply the goods/services and calculating which goods/services they supply.

Globalese NMT:

Due to the legal status of the person, it may lead to this process at the site of the plant, and also a calculation of the completed technician.

This example shows that unfortunately, NMT can produce bad translations too. As I mentioned before, the ratio of good and bad NMT output you will face in a project always depends on the circumstances. Another weak point of NMT is that it currently cannot handle the terminology directly and it acts as a kind of “black box” with no option to directly influence the results.

Reference: https://bit.ly/2hBGsVh

How machine learning can be used to break down language barriers

How machine learning can be used to break down language barriers

Machine learning has transformed major aspects of the modern world with great success. Self-driving cars, intelligent virtual assistants on smartphones, and cybersecurity automation are all examples of how far the technology has come.

But of all the applications of machine learning, few have the potential to so radically shape our economy as language translation. The content of language translation is the perfect model for machine learning to tackle. Language operates on a set of predictable rules, but with a degree of variation that makes it difficult for humans to interpret. Machine learning, on the other hand, can leverage repetition, pattern recognition, and vast databases to translate faster than humans can.

There are other compelling reasons that indicate language will be one of the most important applications of machine learning. To begin with, there are over 6,500 spoken languages in the world, and many of the more obscure ones are spoken by poorer demographics who are frequently isolated from the global economy. Removing language barriers through technology connects more communities to global marketplaces. More people speak Mandarin Chinese than any other language in the world, making China’s growing middle class is a prime market for U.S. companies if they can overcome the language barrier.

Let’s take a look at how machine learning is currently being applied to the language barrier problem, and how it might develop in the future.

Neural machine translation

Recently, language translation took an enormous leap forward with the emergence of a new machine translation technology called Neural Machine Translation (NMT). The emphasis should be on the “neural” component because the inner workings of the technology really do mimic the human mind. The architects behind NMT will tell you that they frequently struggle to understand how it comes to certain translations because of how quickly and accurately it delivers them.

“NMT can do what other machine translation methods have not done before – it achieves translation of entire sentences without losing meaning,” says Denis A. Gachot, CEO of SYSTRAN, a language translation technologies company. “This technology is of a caliber that deserves the attention of everyone in the field. It can translate at near-human levels of accuracy and can translate massive volumes of information exponentially faster than we can operate.”

The comparison to human translators is not a stretch anymore. Unlike the days of garbled Google Translate results, which continue to feed late night comedy sketches, NMT is producing results that rival those of humans. In fact, Systran’s Pure Neural Machine Translation product was preferred over human translators 41% of the time in one test.

Martin Volk, a professor at the Institute of Computational Linguistics at the University of Zurich, had this to say about neural machine translation in a 2017 Slator article:

“I think that as computing power inevitably increases, and neural learning mechanisms improve, machine translation quality will gradually approach the quality of a professional human translator over the coming two decades. There will be a point where in commercial translation there will no longer be a need for a professional human translator.”

Gisting to fluency

One telling metric to watch is gisting vs. fluency. Are the translations being produced communicating the gist of an idea, or fluently communicating details?

Previous iterations of language translation technology only achieved the level of gisting. These translations required extensive human support to be usable. NMT successfully pushes beyond gisting and communicates fluently. Now, with little to no human support, usable translations can be processed at the same level of quality as those produced by humans. Sometimes, the NMT translations are even superior.

Quality and accuracy are the main priorities of any translation effort. Any basic translation software can quickly spit out its best rendition of a body of text. To parse information correctly and deliver a fluent translation requires a whole different set of competencies. Volk also said, “Speed is not the key. We want to drill down on how information from sentences preceding and following the one being translated can be used to improve the translation.”

This opens up enormous possibilities for global commerce. Massive volumes of information traverse the globe every second, and quite a bit of that data needs to be translated into two or more languages. That is why successfully automating translation is so critical. Tasks like e-discovery, compliance, or any other business processes that rely on document accuracy can be accelerated exponentially with NMT.

Education, e-commerce, travel, diplomacy, and even international security work can be radically changed by the ability to communicate in your native language with people from around the globe.

Post language economy

Everywhere you look, language barriers are a speed check on global commerce. Whether that commerce involves government agencies approving business applications, customs checkpoints, massive document sharing, or e-commerce, fast and effective translation are essential.

If we look at language strictly as a means of sharing ideas and coordinating, it is somewhat inefficient. It is linear and has a lot of rules that make it difficult to use. Meaning can be obfuscated easily, and not everyone is equally proficient at using it. But the biggest drawback to language is simply that not everyone speaks the same one.

NMT has the potential to reduce and eventually eradicate that problem.

“You can think of NMT as part of your international go-to-market strategy,” writes Gachot. “In theory, the Internet erased geographical barriers and allowed players of all sizes from all places to compete in what we often call a ‘global economy,’ But we’re not all global competitors because not all of us can communicate in the 26 languages that have 50 million or more speakers. NMT removes language barriers, enabling new and existing players to be global communicators, and thus real global competitors. We’re living in the post-internet economy, and we’re stepping into the post-language economy.”

Machine learning has made substantial progress but has not yet cracked the code on language. It does have its shortcomings, namely when it faces slang, idioms, obscure dialects of prominent languages and creative or colorful writing. It shines, however, in the world of business, where jargon is defined and intentional. That in itself is a significant leap forward.

Reference: https://bit.ly/2Fwhuku

GDPR. Understanding the Translation Journey

GDPR. Understanding the Translation Journey

“We only translate content into the languages of the EU, so we are covered with regards GDPR clauses relating to international transfers.”

Right? Wrong.

The GDPR imposes restrictions on the transfer of personal data outside the European Union (EU), to third-party countries or international organizations. While there are provisions that refer to your ability to do this with the appropriate safeguards in place, how confident are you that you’re not jeopardising GDPR-compliance with outdated translation processes?

Let’s consider the following:

  1. 85% of companies cannot identify whether they send personal information externally as part of their translation process.
  2. The translation process is complex – it isn’t a simple case of sending content from you to your translator. Translating one document alone into 10 languages involves 150 data exchanges (or ‘file handoffs’). Multiply this by dozens of documents and you have a complex task of co-ordinating thousands of highly-sensitive documents – some which may contain personal data.

With different file versions, translators, editors, complex graphics, subject matter experts and in country reviewers the truth is that content is flying back and forth around the world faster than we can imagine. Designed with speed of delivery and time to market in mind these workflows overlook the fact that partners might not share the same compliance credentials.

Where exactly is my data?

Given that we know email is not secure – let us think about what happens when you use a translation portal or an enterprise translation management system.

Once you’ve transferred the content for translation, the translation agency or provider downloads and processes that data on its premises before allocating the work to linguists and other teams (let’s hope these are in the EU and they are GDPR compliant).

Alternatively, the software you have used to share your content may process the data to come up with your Translation Memory leverage and spend – in which case better check your End User Licence Agreement to ensure you know where that processing (and backup) takes place.

After that has happened the content is distributed to the translators to work on. Even if all the languages you translate into are in the EU – are you SURE that your translators are physically located there too?

And what about your translation agency’s project management team? How exactly do they handle files that require Desktop Publishing or file engineering? Are these teams located onshore in the EU or offshore to control costs? If the latter what systems are they using, and how can you ensure no copies of your files are sitting in servers outside of your control?

These are just some of the questions you should be asking now to fully understand where your translation data is located.

What can I do?

If you haven’t already – now is the time to open a conversation with your partner about your data protection needs and what they are doing as a business to ensure compliance. They should be able to tell you exactly which borders your data crosses during the translation process, where it’s stored and what they’re doing to help with Translation Memory management. They should also provide you with a controlled environment that you can use across the entire translation supply chain, so that no data ever leaves the system.

Of course, there are many considerations to take into account when it comes to GDPR. But looking at the complexity of translating large volumes of content – are you still confident that your translation processes are secure?

Reference: https://bit.ly/2vmKKX5

Europe’s New Privacy Regulation GDPR Is Changing How LSPs Handle Content

Europe’s New Privacy Regulation GDPR Is Changing How LSPs Handle Content

GDPR, the General Data Protection Regulation, is soon to be introduced across Europe, and is prompting language service providers (LSPs) to update policies and practices relating to their handling of all types of personal data.

The GDPR comes into effect on 25 May 2018 and supersedes the existing Data Protection Directive of 1995. It introduces some more stringent requirements on how the personal data of EU citizens are treated.

Specifically, LSPs must demonstrate that they are compliant in the way that they handle any type of personal data that at some point flows through their business. Personal data means any information by which a person can be identified, such as a name, location, photo, email address, bank details…the list goes on.

Therefore, LSPs need to ensure that all data, from employee records and supplier agreements to client contact information and content for translation, are handled appropriately.

What personal data do LSPs handle?

Aside from the actual content for translation, an LSP is likely to possess a vast array of personal data including Sales and Marketing data (prospective client details, mailing lists, etc.), existing client data (customer names, emails, POs, etc.), HR and Recruitment data (candidate and employee data including CVs, appraisals, addresses, etc.) and Supplier (freelance) data (bank details, contact details, performance data, CVs, etc.).

In this respect, the challenges that LSPs will face are not significantly different from most other service businesses, and there are lots of resources that outline the requirements and responsibilities for complying with GDPR. For example, the Europa website details some key points, and ICO (for the UK) has a self-assessment readiness toolkit for businesses.

What about content for translation?

Content that a client sends you for translation also may contain personal information. Some of these documents are easy enough to identify by their nature (such as birth, death, marriage certificates, HR records, and medical records), but personal data might be also considered to extend to the case where you receive an internal communication from a customer that includes a quote from the company CEO, for example.

Short-term challenges

It is important to be able to interpret what the GDPR means for LSPs generally, and for your business specifically. The impact of the regulation will become clearer over time, but it throws up some potentially crucial questions in the immediate, such as:

  • What the risks are for LSPs who continue to store personal data within translation memories and machine translation engines;
  • What the implications are for sharing personal data with suppliers outside of the EU / EEA, and specifically in countries deemed to be inadequate with respect to GDPR obligations (even a mid-sized LSP would work with hundreds of freelancers outside the EU);
  • How binding corporate rules can be applied to LSPs with a global presence;
  • Whether obliging suppliers to work in an online environment could help LSPs to comply with certain GDPR obligations

Longer-term considerations

While the GDPR presents a challenge to LSPs in the short-term, it may also impact on the longer-term purchasing habits within the industry.

For example, if LSPs are penalized for sharing personal data with freelancers located within inadequate countries (of which there is a long list), LSPs could be forced to outsource translation work within the EU / EEA / adequate countries only or even insource certain language combinations entirely, potentially driving up the cost of translation spend for some languages.

Or, if a client company is penalized for sharing personal data with a subcontractor (i.e. an LSP or freelancer) without the full knowledge and consent of the person the information relates to (known as the data subject), will they be more inclined to employ alternative buying models for their language needs: e.g. to source freelancers directly or via digital marketplaces, or implement in-house translation models of their own?

Get informed

Although most LSPs are well-acquainted with data privacy, there are a lot of unknowns around the impact of GDPR, and LSPs would be wise to tread especially carefully when it comes to handling personal data, in particular post-25 May.

Perhaps the noise around GDPR turns out to be hot air, but with companies in breach of the regulation facing possible penalties that the GDPR recommends should be “effective, proportionate and dissuasive”, it is essential to get informed, and quickly.

Reference: https://bit.ly/2Jwh9g6

How Lingotek Uses AI to Optimize Vendor Management

How Lingotek Uses AI to Optimize Vendor Management

Language Services Vendor Management is a complex management task. It requires vetting multiple language services providers (LSPs), requesting multiple bids and comparing different rate structures. It can include literally hundreds of projects to monitor and manage to ensure on-time delivery. Adding to the complexity, LSPs typically use  several different computer-assisted translation (CAT) tools and maintain multiple linguistic assets in various offline locations. How well translation is managed has a direct effect on the company’s globalization goals and its ability to execute an agile go-to-market strategy.

No one makes vendor management easier than Lingotek. Our groundbreaking artificial intelligence (AI)-driven app inside our industry-leading translation management system (TMS) is a cost-efficient localization platform that simplifies vendor management, enhances efficiency, accelerates delivery, and optimizes budgets and costs to reduce your translation spend.

What is Artificial Intelligence?

Artificial Intelligence (AI) is simply technology that learns. AI uses data and experience to perform tasks that would otherwise require human intelligence and effort. When applied to Vendor Management, it creates a foundation for trigger-based automation, rule-driven systems, and data collection.

How does Lingotek use AI to optimize vendor management?

Lingotek continues to spearhead innovation in the translation industry with a Vendor Management app that brings AI-driven automation and multilingual business intelligence to translation management. The entire process for managing vendors: vendor selection, tracking costs and spending, vendor performance is now easier and more automated. With this data, organizations can easily and repeatedly select vendors who provide the highest translation quality and who consistently deliver jobs on time.

Integrated & automated vendor selection

The Vendor Management app simplifies and consolidates the process for requesting quotes, setting rates and pricing, choosing vendors, managing deadlines, tracking spending, and measuring translator quality and performance. The dashboard displays all of the information needed for tracking and evaluating which vendors are providing the highest quality translation and meeting deadlines. This gives project managers insights to better manage workloads and resources for maximum throughput.

  • Automatic vendor assignment based on language, industry, timeline, and more.
  • Automated bid requests, rate charts & invoicing.
  • Monitor costs and billing information within the TMS.

Centralized tracking of rates, costs & spending

The Vendor Management app automates many of the steps required for creating a language services purchase order and to closely track translation spending. The app also tracks the leveraging of translation memories (TM) to gauge the efficient reuse of linguistic assets across the enterprise. At-a-glance rate charts for quick reference of:

  • Integrated cost reporting inside the TMS.
  • Total translation expenses by date, job, or vendor.
  • Aggregation of data to simplify invoice creation.

Automatic cost calculation

Lingotek’s vendor management includes auto-calculation of costs, even when specific jobs have been skipped or cancelled. A project manager can manually skip or cancel a phase, target, or entire document.

With the active monitoring offered by our Intelligent Workflows, jobs can also be auto-skipped or auto-cancelled in order to ensure on-time delivery. When this happens, our AI-driven Vendor Management system is able to proactively alert vendors of the skipped and/or cancelled job, ensure that additional work cannot be performed on those skipped and/or cancelled jobs, and then automatically calculate the the costs for the work that was completed before the job was cancelled.

This makes invoicing a breeze, as project managers and vendor managers no longer have to worry about notifying vendors of changes made to the project mid-stream, or figure out how much work was done after the fact in order to manually calculate their costs.

Intelligence & insight to optimize your supply chain

Get more data-driven insight and control over your localization supply chain. The dashboard displays tracking and evaluating information on vendors, so you can easily select vendors who provide the highest translation quality and consistently deliver jobs on time. This gives you much-needed insight to better manage workloads and resources for maximum throughput.

  • Vendor-specific intelligence.
  • Evaluate vendor performance & quality through SLA compliance metrics.
  • Monitor project delivery & efficiency by vendor.
  • Get key metrics on costs, turnaround time, word counts, missed deadlines.

As the technology improves, we recommend that all providers review their operations to learn where they could take best advantage of AI.

–Common Sense Advisory, “The Journey to Project Management Automation”

Discover the Benefits of Lingotek’s AI-Driven Vendor Management

The new Vendor Management app gives enterprise localization managers, vendor managers, and project managers revolutionary new tools for managing multiple language services providers (LSPs) and projects. Automating vendor management provides critical operational efficiency to enable more scalable globalization strategies and to optimize your localization supply chain to create a more cost-efficient localization network.

Lingotek’s AI-driven Vendor Management can reduce the need for project managers to perform routine, automated tasks. Instead, they can use that time for solving problems that AI can’t solve. When you implement better process automation, that leaves time for project managers to perform tasks that are more valuable to the organization. They can focus their time on exception management–problem solving and responding to urgent issues.

Reference: https://bit.ly/2wONm0C

A New Way to Measure NMT Quality

A New Way to Measure NMT Quality

Neural Machine Translation (NMT) systems produce very high quality translations, and are poised to radically change the professional translation industry. These systems require quality feedback / scores on an ongoing basis. Today, the prevalent method is via Bilingual Evaluation Understudy (BLEU), but methods like this are no longer fit for purpose.

A better approach is to have a number of native speakers assess NMT output and rate the quality of each translation. One Hour Translation (OHT) is doing just that: our new NMT index is released in late April 2018 and fully available for the translation community to use.

A new age of MT

NMT marks a new age in automatic machine translation. Unlike technologies developed over the past 60 years,  the well-trained and tested NMT systems that are available today,  have the potential to replace human translators.

Aside from processing power, the main factors that impact NMT performance are:

  •      the amount and quality of initial training materials, and
  •      an ongoing quality-feedback process

For a NMT system to work well, it needs to be properly trained, i.e. “fed” with hundreds of thousands (and in some cases millions) of correct translations. It also requires feedback on the quality of the translations it produces.

NMT is the future of translation. It is already much better than previous MT technologies, but issues with training and quality assurance are impeding progress.

NMT is a “disruptive technology” that will change the way most translations are performed. It has taken over 50 years, but machine translation can now be used to replace human translators in many cases.

So what is the problem?

While NMT systems could potentially revolutionize the translation market, their development and adoption are hampered by the lack of quality input, insufficient means of testing the quality of the translations and the challenge of providing translation feedback.

These systems also require a lot of processing power, an issue which should be solved in the next few years, thanks to two main factors. Firstly, Moore’s law, which predicts that processing power doubles every 18 months, also applies to NMT, meaning that processing power will continue to increase exponentially. Secondly, as more companies become aware of the cost benefit of using NMT, more and more resources will be allocated for NMT systems.

Measuring quality is a different and more problematic challenge. Today, algorithms such as BLEU, METEOR, and TER try to predict automatically what a human being would say about the quality of a given machine translation. While these tests are fast, easy, and inexpensive to run (because they are simply software applications), their value is very limited. They do not provide an accurate quality score for the translation, and they fail to estimate what a human reviewer would say about the translation quality (a quick scan of the text in question by a human would reveal the issues with the existing quality tests).

Simply put, translation quality scores generated by computer programs that predict what a human would say about the translation are just not good enough.

With more major corporations including Google, Amazon, Facebook, Bing, Systran, Baidu, and Yandex joining the game, producing an accurate quality score for NMT translations becomes a major problem that has a direct negative impact on the adoption of NMT systems.

There must be a better way!

We need a better way to evaluate NMT systems, i.e. something that replicates the original intention more closely and can mirror what a human would say about the translation.

The solution seems simple: instead of having some software try to predict what a human would say about the translation, why not just ask enough people to rate the quality of each translation? While this solution is simple, direct, and intuitive, doing it right and in a way that is statistically significant means running numerous evaluation projects at one time.

NMT systems are highly specialized, meaning that if a system has been trained using travel and tourism content, testing it with technical material will not produce the best results. Thus, each type of material has to be tested and scored separately. In addition, the rating must be done for every major language pair, since some NMT engines perform better in particular languages. Furthermore, to be statistically significant, at least 40 people need to rate each project per language, per type of material, per engine. Besides that, each project should have at least 30 strings.

Checking one language pair with one type of material translated with one engine is relatively straightforward: 40 reviewers each check and rate the same neural machine translation consisting of about 30 strings. This approach produces relatively solid (statistically significant) results, and repeating it over time also produces a trend, i.e. making it possible to find out whether or not the NMT system is getting better.

The key to doing this one isolated evaluation is selecting the right reviewers and making sure they do their job correctly. As one might expect, using freelancers for the task requires some solid quality control procedures to make sure the answers are not “fake” or “random.”

At that magnitude (one language, one type of material, one NMT engine, etc), the task is manageable, even when run manually. It becomes more difficult when an NMT vendor, user, or LSP wants to test 10 languages and 10 different types of material with 40 reviewers each. In this case, each test requires between 400 reviewers (1 NMT engine x 1 type of material x 10 language pairs x 40 reviewers) and 4,000 reviewers (1 NMT engine x 10 types of material x 10 language pairs x 40 reviewers).

Running a human based quality score is a major task, even for just one NMT vendor. It requires up to 4,000 reviewers working on thousands of projects.

This procedure is relevant for every NMT vendor who wants to know the real value of their system and obtain real human feedback for the translations it produces.

The main challenge is of course finding, testing, screening, training, and monitoring thousands of reviewers in various countries and languages — monitoring their work while they handle tens of thousands of projects in parallel.

The greater good – industry level quality score

Looking at the greater good,  what is really needed is a standardised NMT quality score for the industry to employ, measuring all of the various systems using the same benchmark, strings, and reviewers, in order to compare like for like performance. Since the performance of NMT systems can vary dramatically between different types of materials and languages, a real human-based comparison using the same group of linguists and the same source material is the only way to produce real comparative results. Such scores will be useful both for the individual NMT vendor or user and for the end customer or LSP trying to decide which engine to use.

To produce the same tests on an industry-relevant level is a larger undertaking. Using 10 NMT engines, 10 types of material, 10 language pairs and 40 reviewers, the parameters of the project can be outlined as follows:

  •      Assuming the top 10 language pairs are evaluated, ie EN > ES, FR, DE, PT-BR, AR, RU, CN, JP, IT and KR;
  •      10 types of material – general, legal, marketing, finance, gaming, software, medical, technical, scientific, and tourism;
  •      10 leading (web-based) engines – Google, Microsoft (Bing), Amazon, DeepL, Systran, Baidu, Promt, IBM Watson, Globalese and Yandex;
  •      40 reviewers rating each project;
  •      30 strings per test; and
  •      12 words on average per string

This comes to a total of 40,000 separate tests (10 language pairs x 10 types of material x 10 NMT engines x 40 reviewers), each with at least 30 strings, i.e. 1,200,000 strings of 12 words each, resulting in an evaluation of approximately 14.4 million words. This evaluation is needed to create just one instance (!) of a real, comparative, human-based NMT quality index.

The challenge is clear: to produce just one instance of a real viable and useful NMT score, 4,000 linguists need to evaluate 1,200,000 strings equating to well over 14 million words!

The magnitude of the project, the number of people involved and the requirement to recruit, train, and monitor all the reviewers, as well as making sure, in real time, that they are doing the job correctly, are obviously daunting tasks, even for large NMT players, and certainly for traditional translation agencies.

Completing the entire process within a reasonable time (e.g. less than one day), so that the results are “fresh” and relevant makes it even harder.

There are not many translation agencies with the capacity, technology, and operational capability to run a project of that magnitude on a regular basis.

This is where One Hour Translation (OHT) excels. They have recruited, trained, and tested thousands of linguists in over 50 languages, and already run well over 1,000,000 NMT rating and testing projects for our customers. By the end of April 2018, they published the first human-based NMT quality index (initially covering several engines and domains and later expanding), with the goal of promoting the use of NMT across the industry.

A word about the future

In the future, a better NMT quality index can be built using the same technology NMT is built on, i.e. deep-learning neural networks. Building a Neural Quality system is just like building a NMT system. The required ingredients are high quality translations, high volume, and quality rating / feedback.

With these ingredients, it is possible to build a deep-learning, neural network based quality control system that will read the translation and score it like a human does. Once the NMT systems are working smoothly and a reliable, human based, quality score/feedback developed, , the next step will be to create a neural quality score.

Once a neural quality score is available, it will be further possible to have engines improve each other, and create a self-learning and self-improving translation system by linking the neural quality score to the NMT  (obviously it does not make sense to have a closed loop system as it cannot improve without additional external data).

With additional external translation data, this system will “teach itself” and learn to improve without the need for human feedback.

Google has done it already. Its AI subsidiary, DeepMind, developed AlphaGo, a neural network computer program that beat the world’s (human) Go champion. AlphaGo is now improving, becoming better and better, by playing against itself again and again – no people involved.

Reference: https://bit.ly/2HDXbTf