/
Best Engine

Best Engine

Best Engine is a powerful tool designed to evaluate and compare stock and custom models from various providers, helping you identify the optimal fit for your specific data requirements.

Best Engine operates through three key layers:

  • Data Preparation: Cleans the TMX file prior to processing, ensuring only valid and meaningful real-world data is used.

  • Model Evaluation: Compares translations across models using BLEU and WER metrics for objective performance assessment.

  • Results Visualization: Delivers a visual output highlighting the two top-performing models, accompanied by a detailed final report.

 

Running an Evaluation

To begin using Best Engine:

  1. Click on Best Engine in the menu on the left.

  2. Click on “Upload a TMX File” and navigate to the TMX file you want to use for testing

Your TMX need to have UTF-8 encoding and have a maximum size of 5 MB.

  1. Choose the source and target languages. Use non-locale variants (e.g., English to German).

  2. Optionally select a domain. The default option is "None" and domains will not influence the evaluation in any way, they will only help in categorising the projects in the evaluation list.

    image-20251006-155511.png

 

  1. Select the providers for comparison by scrolling down and selecting the checkboxes adjacent to each option.

image-20251006-160203.png

  1. Click on “Start Evaluation” to begin the comparison process.

  2. You can monitor the progress. Once the TMX cleaning and translation steps are complete, the status indicator will turn green.

image-20251006-161110.png

Expand the steps to reveal comprehensive details of the evaluation process and access downloadable data.

db353d56-e1f2-48a6-81cd-6d0d66a16fb1.png

  1. To create the ranking report and graph, click on “Continue”.

  2. Upon completion, Best Engine will display the top two providers and offer the option to download a report with detailed results for each segment.

image-20251006-161633.png

 

 

FAQ

  1. Connect your custom model to the console. See Connections | Custom Engines

  2. Select your language pair

  3. Your custom model will appear in the list. Languages need to match exactly the ones set when connecting the model.

TMX files should:

  • have UTF-8 encoding

  • be max 1000 segments. If your TMX is longer, it will be cut in the first step of the process, but the evaluation might take longer.

  • Be max 5 mb