Validating a Model

Now that you have trained a model, you need to validate that it is performing the way it was intended and that it is solving the problem it was designed to solve.

Validation should also be done routinely after deployment (for example, every three months) as part of the operations and monitoring process.

Another scenario where you would want to perform additional validation is when there are changes in hardware or environment conditions that could affect your model’s results. For example, a change in lighting conditions, a change in camera position, or an addition of a new camera.

Follow these steps to validate your models:

1. Return to the My Workspace tab. If you are not already in the AI models section, click AI Models in the left navigation bar. In the Model Names column, click the expansion button to the left of a model to view its details.

   

   1. For this example, we are comparing model accuracy for variants of a model trained using YOLOX and ATSS. The ATSS model shows a higher percentage of accuracy than YOLOX indicating that it is a better option of the two.

      

      

   2. Now, let us compare two variations of a model trained with the same topology but a different dataset.

      

      Click View Data Stats for both models. For this example, we are interested in the “defect” label. We can see that the detection rate is higher for version 2 shown on the right.

      

2. At this point you may have enough information to decide whether to deploy a model or retrain it. You can also continue with the validation flow to obtain more information.

3. Click Validate under the Actions column.

   

4. The Upload dataset window appears. You can upload a dataset of images and an annotation file from these locations: The Edge Platform file system, cloud services, or your local system. Currently, only the Common Objects in Context (COCO) dataset format is supported.

   

   After the images and annotation file are uploaded, click Validate to continue.

   

5. In the Validated result page, you will see more details such as an F1 score (a standard machine learning evaluation metric), a circle graph showing mean average precision, and a download button for an Excel* file with granular data.

   

6. You can click on Download Result to download the Excel file to your computer. The file contains raw data (prediction dump) with details such as image name, category, predicted label, bounding box and score.