How To Set Up a Language Quality Measurement System

The six phases for setting up a robust quality system:

1. Design
   
   • - Collect a corpus of good and bad translations
     
   • - Analyze the corpus to identify critical-to-quality errors (error definition)
     
   • - Decide what to measure (error categorization)
     
   • - Assign a weight to various types of errors (severities)
     
   • - Define an error threshold
     


2. Calibration
   
   • - Pilot,
     
   • - test, and
     
   • - adjust
     until the system works in an objective, repeatable, and reproducible way
     


3. Sampling
   
   • - Sample selection criteria (e.g. random, systematic)
     
   • - Sample size, confidence intervals, margins of error
     
   • - Cost considerations (find the point of diminishing returns)
     


4. Measurement
   
   • - Calculation of a Translation Quality Index (TQI)
     
   • - Evaluation must be
     
     • * repeatable,
       
     • * reproducible,
       
     • * objective
       
   • - Use of independent auditors
     
   • - Provide formal training
     


5. Statistical Analysis
   
   • - Use of control charts
     
   • - Investigate results
     
   • - Rule out special causes
     


6. Process Improvement
   
   • - Take corrective actions
     
   • - Compare the TQI values before and after a process change to check for actual process improvement