Quality Doesn't Matter (as a differentiator)?

I didn't go to the ATA conference in San Francisco, so I missed Renato Beninatto's presentation "Quality Still Doesn't Matter".

I recommend reading the presentation (although of course it can only contain the gist of what the actual presentation was like): behind the provocative title, I think it has interesting things to say.

Results of Translation QA Tool Survey

Tamara Lukyanova and Julia Makoushina, of Palex Languages & Software, have concluded their work on the interesting survey of translation QA tools we announced here in August.

From August through September 2007 Palex performed a translation community survey to evaluate acceptance of translation quality assurance tools by the translation community, to define where such tools currently are, their advantages and disadvantages and to visualize their future capabilities and their role in the translation process. In parallel with the survey, Palex tested and benchmarked translation quality assurance tools currently available in the market.

Julia has recently given a presentation ("Translation Quality Assurance Tools: Current State and Future Approaches") at the 29th Translating and the Computer conference that took place on 29-30 November 2007 in London. In addition to the presentation (which can be downloaded from the previous link), Julia has also written a paper with the same title. Both paper and presentation are very interesting and well worth downloading (you can do so from the previous links) and reading.

Translation QA Tool Survey

Tamara Lukyanova and Julia Makoushina, of Palex Languages & Software, have set up an interesting survey about translation QA tools.

From 1 August through September 28, 2007 Palex will perform a translation community survey to evaluate acceptance of translation quality assurance tools by the translation community, to define where such tools currently are, their advantages and disadvantages and to visualize their future capabilities and role in translation process.

[...]

In parallel with the survey, Palex is going to test and benchmark translation quality assurance tools currently available in the market. The survey and benchmarking results will be presented at 29th Translating and the Computer conference that will tale place on 29-30 November 2007 in London, as well as in other industry publications.

According to Tamara and Julia, if you respond to the survey and provide your e-mail address, you will be able to receive the survey analysis as soon as the survey is closed.

Please feel free to point other colleagues to the questionnaire.

To access the survey, click here (or on the link at the beginning of this post).

Translation QA Tools

I've just added a page to TranslationQuality.com to provide links to QA Distiller, Error Spy, and XBench, three tools designed to perform QA checks on translation files.

As soon as I can, I will put up a table to compare the features offered by these tools.

If you know of any other translation QA tools, please let us know.

Update

Also added to the page a link to BlackJack.

ApSIC’s XBench: New QA Functions Help Providing Consistent Translations

Many translators have been using XBench, a free utility created by ApSIC (a Spanish localization company) as a handy program to check glossaries or exported translation memories. Its main use was probably as a convenient and simple tool to search the Microsoft Glossaries.

ApSIC has recently released an upgrade to the program; though it looks like a minor upgrade (from version number 2.6 to 2.7), it is in fact a major improvement, which adds many more file formats (including Trados and SDLX bilingual files), and, above all, powerful quality assurance tools.

The new functionality permit to define files as “key terms” or as “ongoing translation”; the two things are linked: the ongoing translations files are checked against the key terms glossaries, and any discrepancy is noted in the QA screen. In addition to that, the new QA functions warn of various other potential problems, including numeric or tag mismatches, and permit to run checklists against the translated files (I believe this could be useful, for example, to check against specific types of problems).

The new features make the program more complicated (though the basic glossary search functionality remains as simple as before), but they are clearly explained in the thorough help file.

All in all, XBench is now a program that should prove very useful to almost all translators, and at a price that is impossible to beat.

New page of translation quality links at TranslationQuality.com

We have just added a new page of
interesting links about translation quality at TranslationQuality.com, our web site devoted to translation quality issues.

Developing and using a translation quality index

Our new article "Developing and using a translation quality index" has just been published in the July/August 2006 issue of MultiLingual.
The article is not freely available online (though you can get to an excerpt following the "current issue" link on MultiLingual's home page), however, if you are interested in reading it, please contact me.

Translation Quality Measurement Approaches

Six approaches to measuring translation quality is a short article published several years ago on the Medical Design Online web site (the article is still available).

The six approaches mentioned are:

1. Society of Automotive Engineers (SAE) J2450
2. Localisation Industry Standard Association (LISA) QA Model
3. ForeignExchange Translations' Multilingual Compliance Process
4. American Translators Association accreditation program
5. Deutsches Institut für Normung DIN 2345
6. Six sigma quality initiative

The article contains some further details about these differing approaches to quality.

Standards and Models

Better Localization has an interesting article (Standards and models, why should we care?) about how the application of various standards and models would be beneficial in the localization industry.

They are not goals, but tools to achieve goals

The standards described briefly in teh article are ISO 9001:2000, CMMI, BSC, PCMM, and their integration.

ISO 9000 Certification, and Other Methods of Certifying the Quality of LSPs

A few days ago Common Sense Advisory published an article (Looking for Language Service Providers in All the Right Places) on how customers may make sure that LSPs (Language Service Providers) meet their business needs.

...training and certification are good ideas that we believe every buyer ultimately will demand of their suppliers.

Among the interesting things mentioned in the article there is the fact that only 10% of LSPs have ISO 9000 certification.

The article ends up promoting Common Sense Advisory's own certification service for LSPs, which itself looks interesting:

We take a financial auditor's approach to the question of certification -- such firms check a company's compliance to Generally Accepted Accounting Principles (GAAP). Similarly, our 24x7 program looks for "a" process that is used consistently and constantly throughout an LSP's organization. That process can be ISO, defined by a client, internally developed, or any combination of these.

Our Article to Appear in Multilingual

The title of the article is "Translation Quality Measurement: Using the Translation Quality Index to assess the quality of translations", and deals with such questions as:

• Why measure translation quality?


• Why are translations so difficult to evaluate? What methods are available to help assess translation quality?


• What is the Translation Quality Index (TQI)?


• Why measure errors?
  

The article is scheduled to appear in the June 2006 issue of Multilingual, which should also contain other articles about translation quality.

UPDATE

(4/26/06)
The article will actually appear in the July/August issue, not the June issue.

CEN Approves Translation Quality Specification

Common Sense Advisory has an interesting article about the approval, by CEN (the European Committee for Standardization), of EN 15038 quality standard for translation service providers.

The article also compares EN 15038 with ASTM 15.48, then goes on and criticizes the antiquated procedures that were followed in drafting the standard, and above all the lack of transparency in these procedures.

How To Set Up a Language Quality Measurement System

The six phases for setting up a robust quality system:

1. Design
   
   • - Collect a corpus of good and bad translations
     
   • - Analyze the corpus to identify critical-to-quality errors (error definition)
     
   • - Decide what to measure (error categorization)
     
   • - Assign a weight to various types of errors (severities)
     
   • - Define an error threshold
     


2. Calibration
   
   • - Pilot,
     
   • - test, and
     
   • - adjust
     until the system works in an objective, repeatable, and reproducible way
     


3. Sampling
   
   • - Sample selection criteria (e.g. random, systematic)
     
   • - Sample size, confidence intervals, margins of error
     
   • - Cost considerations (find the point of diminishing returns)
     


4. Measurement
   
   • - Calculation of a Translation Quality Index (TQI)
     
   • - Evaluation must be
     
     • * repeatable,
       
     • * reproducible,
       
     • * objective
       
   • - Use of independent auditors
     
   • - Provide formal training
     


5. Statistical Analysis
   
   • - Use of control charts
     
   • - Investigate results
     
   • - Rule out special causes
     


6. Process Improvement
   
   • - Take corrective actions
     
   • - Compare the TQI values before and after a process change to check for actual process improvement
   
   

Writing for Translation

An aspect of translation quality that is often not mentioned, is how much the quality of the SL affects the quality of the translation.

Courses on "Writing for Translation" such as this one are often popular, and, when done right, also useful, as technical writers may learn why, for example, concatenated strings are so much trouble for us.

One thing I saw when I worked in a large business software company, however, was that technical writers often resist any attempt to introduce controlled English, to limit their terminological choices, to introduce stringent editing rules, or, more in general, anything that they perceive as limiting their freedom and creativity.

"About Translation" Locked by BlogSpot

I have another blog on BlogSpot, About Translation, where I write more in general about translation and things of interest for professional tanslators.

Unfortunately, since last week my blog has been locked by Blogger, as their algorithms have (wrongly!!!) deemed it a "spam blog" (no idea why: it is clearly a legitimate blog, but until some human person takes a look, it remains locked)... so while the blog is still up, I cannot post anything new.

If the situation persists, I'll move the blog to some other service.

UPDATE: About Translation has finally been checked by a human member of the support team, white listed, and unlocked.

Brave Comment on the Blog

We have just received a comment on our blog, from "Anonymous":

That's rich, a blog about quality in translation operated by a Lionbridge employee. Lionbridge is a fraudulent company that doesn't pay its translators.

To which I have a few things to say:

1. This blog is completely independent from Lionbridge; it was set up to complement our web site TranslationQuality.com with an area where we could discuss issues related to translation in a more dynamic way than on the main web site (where we make available more information about our work on translation quality, such as our presentations and articles).
   
2. Franco is indeed a Lionbridge employee. I am not: I have my own small translation company, and we provide our services to several larger translation companies. Among the services we provide on request is translation quality assessment services, including helping other companies improving their translation quality systems.
   
3. This information is clearly available both on the "Profiles" section of this blog and on our main web site.
   
4. I suggest that if "Anonymous" has a complaint against Lionbirdge, he should contact Lionbridge directly, and, if not satisfied, take the appropriate legal steps. On the other hand, this blog is not an appropriate venue for such complaints.
   
5. While Franco and I are clearly named in our blog, "Anonymous" has not the courage of signing his message. That, to me, says all there is to say about the merit of his complaint.
   

Any further anonymous abusive messages will be deleted from this blog without comment.

Revision, Editing and Proofreading

The February issue of the ATA Chronicle has just arrived, and with it an interesting article on translation quality control: "A Second Pair of Eyes: Revision, Editing and Proofreading", by S. Alexandra Russell-Bitting.

After introducing herself and her experience, the author devotes sections of her article to Quality Control in Translation, Revision, Editing, Proofreading, and Managing Quality.

Apart from other things, the article could be very useful in that it differentiates between activities that are often lumped together, confused with each other, or misnamed.

In this case, "Revision" is defined as "checking a translation for accuracy and style", going through the text sentence by sentence and comparing SL and TL.

"Editing" is described as "more creative work than revision [where] you have the freedom to make improvements in the text for readability"

"Proofreading", finally, according to Russell-Bitting is "a final check before publication".

When I am asked by a translation company to "proof" or "edit" a translation, I always have to ask what, exactly, they want me to do: what's proofreading to someone is called editing by a others, and revision is confused with both things. I hope this article may help differentiate between these important translation quality control activities.

A Simple QC Technique for Bilingual Files

One of the most insidious kind of errors happens when a negation is translated in the affirmative, or vice-versa. Insidious because sometimes such errors are difficult to spot, as the sentences that contain them, on their own, probably still make sense.

However, when translating using an environment, such as TM program, which produces bilingual files, there is a simple quality control technique that, while not 100% effective, still helps avoiding this type of errors. The following example works for the English-Italian language pair, but it could easily be adapted to other languages:

1. Once the translation (or editing) is complete, search for word "not" throughout the bilingual file
2. For each occurrence of the word found, check the corresponding Italian segment, to make sure that the negative meaning is preserved
   
3. Once completed the search for the English "not", repeate the search for the Italian "non", so as to make sure that a negative meaning has not cropped where the English instead has an affermative sentence
   

Of course this is not 100% foolproof, but it can be improved by adding other relevant words (such as "cannot", "failed", "impossible", etc.) to the search.

Why Translation Quality Measurement?

You cannot manage what you cannot (or do not) measure; yet, this is precisely what most translation quality control processes try to do.

A Method for Predicting the Productivity of Machine Translation

Better Localization has recently published an interesting article on how to predict the productivity of an MT and post-editing process.

The article also deals by necessity with the translation quality level of machine translation, as the productivity that can be achieved during post-editing heavily depends on the quality of the MT output.

I have some issues with some of the things the article says (for instance that "testers and cannot give reliable quantitative results in a cost-effective manner.": I believe that the work Franco and I have done on translation quality measurement is cost effective and could be adapted to MT as well, but the article is very well worth reading, and suggests a novel approach that looks promising.