What is custom training/custom trained model?

The process of providing language assets (bilingual data) as teaching data to a machine translation (MT) learning service for machine learning is called "Custom training" (sometimes called "Adaptation" or "Relearning"). As the result of custom training, a custom trained model is created. Once the custom training has been completed, the custom trained model can be used in Quick MT/PE and LAC (Custom Glossary Model Registration) of this system.

Information in Custom Trained Model Details screen

Detailed data of the custom trained model is displayed on this screen. You can only change the following items.

• Custom Trained Model Name: An arbitrary string of each model. It is displayed in the following places. The default value is “Custom Trained Model_YYYYMMDDhhmmss" whose end is the timestamp (JST) when the registration of the custom trained model started. (Example: “Custom Trained Model_20220201152040" if it starts at Feb 1, 2022, 15:40 JST)
  
  • Quick MT, Quick PE (Project Details screen):
    
    • MT engine "Google Translate": Options in the pull down list of Optional settings [Custom Trained Model]
    • MT engine "Kawamura NMT": Options in the pull down list of Required settings [General Model/Custom MT Model]

• • In LAC's Custom MT Model Management (Custom glossary setting): A screen for selecting the target model to apply a glossary
• Model Status: Specifies whether to be used in Quick MT/PE and LAC. The default value is “Enabled”. Set “Disabled” for a custom trained model that are not used in Quick MT/PE and LAC.
• Comment: You can add any information about the custom trained model.
  
  

Also, at the bottom of the screen, the results of the model evaluation (if the model evaluation has been performed) and information on the language assets used for the custom training are displayed.

Model evaluation for custom trained models

On this screen, you can perform model evaluation of the custom trained model. When you press the [Model Evaluation] button at the bottom of the screen, you can upload a bilingual file (which contains bilingual data of the source text and the correct translation, file format: xliff, csv, tsv, maximum 50,000 characters) for the model evaluation.

As a result of the model evaluation, the following four indicators are calculated as numerical values, and you can see the changes before and after the custom training as follows. The former is the value from the general model (*) of the MT learning service used for the custom training, and the latter is the value from the custom trained model. If the value of BLUE, NIST, and RIBES changes higher, and the value of WER changes lower, it can be interpreted that an effective customization has been implemented by the custom training. You can also check the detailed meaning of each indicator in this blog post.

• BLEU
• NIST
• RIBES
• WER