Thursday, March 23, 2006

How To Set Up a Language Quality Measurement System

The six phases for setting up a robust quality system:

  1. Design
    • - Collect a corpus of good and bad translations
    • - Analyze the corpus to identify critical-to-quality errors (error definition)
    • - Decide what to measure (error categorization)
    • - Assign a weight to various types of errors (severities)
    • - Define an error threshold

  2. Calibration
    • - Pilot,
    • - test, and
    • - adjust
      until the system works in an objective, repeatable, and reproducible way

  3. Sampling
    • - Sample selection criteria (e.g. random, systematic)
    • - Sample size, confidence intervals, margins of error
    • - Cost considerations (find the point of diminishing returns)

  4. Measurement
    • - Calculation of a Translation Quality Index (TQI)
    • - Evaluation must be
      • * repeatable,
      • * reproducible,
      • * objective
    • - Use of independent auditors
    • - Provide formal training

  5. Statistical Analysis
    • - Use of control charts
    • - Investigate results
    • - Rule out special causes

  6. Process Improvement
    • - Take corrective actions
    • - Compare the TQI values before and after a process change to check for actual process improvement


No comments: