What is custom training/custom trained model?

The process of providing language assets (bilingual data) as teaching data to a machine translation (MT) learning service for machine learning is called "Custom training" (sometimes called "Adaptation" or "Relearning"). As the result of custom training, a custom trained model is created. Once the custom training has been completed, the custom trained model can be used in Quick MT/PE and LAC (Custom Glossary Model Registration) of this system.

Information in Custom Trained Model List screen

On this screen, a list of custom trained models registered in the organization is displayed. You can also search for a custom trained model by specifying conditions. To add a new custom trained model, click on "+ Register" button at the top of the screen.

• Search criteria area
  
  • Start Date: The start date of the custom training  process (UTC)
  • Machine Translation Learning Service: Specifies the service used for custom training of the model. The following three types of MT learning services can be used in this system.
    
    • Google AutoML
    • Kawamura NMT
  • Status: See the section below for changes in the status of the custom trained models. In Quick MT/PE and LAC, only models with a status of “Custom Training Completed” are available.
  • Model Status: Only models with "Enabled" status are available in Quick MT/PE and LAC.
• List of custom trained models
  
  • Custom Trained Model Name: Click to go to Custom trained model details screen.
  • MT Learning Service
  • Status
  • Enabled: "○" is displayed when the model is enabled.
  • Start Date/Time, End Date/Time: Start and end date/time of the custom training.
  • Evaluation Value: If you have already performed a model evaluation for the custom trained model, the BLEU value is displayed out of the four quality indicators. This field is blank if the model evaluation has not been performed.
  • Comment: Displays the comments entered in Custom trained model details screen.

Status of a custom trained model

The status of a custom trained model can be one of the following 11 types.

Deletion of custom trained models

You can select one or more models and delete them at once. Select the check box of the target model from the list, and click [Delete] icon in Batch Processing menu. Please note that deleting a model has the following effects.

• LAC (Custom Machine Translation Management Menu): The custom trained model is deleted and no longer appears in the list. In addition, the custom glossary model created by applying a glossary to the custom trained model will also be deleted at the same time, and will not be displayed in the list of the custom glossary models.
• Quick MT: When data for a custom trained model is deleted, it no longer appears in the pull-down list of the available custom trained models for each MT engine. Also, if there is a custom glossary model created by applying a glossary to the custom trained model, it also no longer appears in the pull-down list of the available custom glossary models for each machine translation engine. 
• Quick PE: When data for a custom trained model is deleted, it no longer appears in the pull-down list of the available custom trained models for each MT engine. Also, if there is a project that selects the deleted custom trained model, it is possible to reopen File Editor screen of the files in that project, but re-translation will result in an error. In addition, if there is a custom glossary model created by applying a glossary to the custom trained model, it also no longer appears in the pull-down list of the available custom glossary models for each machine translation engine.

Model evaluation of custom trained models

Click the name of each custom trained model in the list to go to the custom trained model details screen where you can evaluate the model. [Evaluation Value] column in the list shows only the BLEU value out of the four metrics. You can check all evaluation values on the custom trained model details screen.

Before the “→” sign is the evaluation value of the general model (*) of MT learning service used for custom training, and the latter is the evaluation value of the custom trained model. If the values of BLUE, NIST, and RIBES changes larger, and WER changes smaller, it can be interpreted that effective customization has been implemented by custom training. You can also check the detailed meaning of each metric in this blog post (Japanese).

• BLEU
• NIST
• RIBES
• WER