Convert and Optimize Model with OpenVINO™ Toolkit

To work with Deep Learning Workbench, you need to obtain a model in the Intermediate Representation (IR) format. This step converts and optimizes models to the IR format.

You need to specify the precision at which model weights should be stored for the conversion to the IR format. You can select either 32-bit (FP32) or 16-bit (FP16) floating-point values.

FP16 is selected for this example because it maintains good accuracy and provides a smaller footprint.

For this step, the process will vary slightly between models downloaded from the Open Model Zoo and custom downloaded models.

For models imported from Open Model Zoo select FP16 and click Convert Model to continue:

For custom models, it is imperative to specify the desired input shape, addressing the model data layout. In deep learning models, the NHWC (Batch size-Height-Width-Channels) and NCHW (Batch size-Channels-Width-Height) data layout formats play a crucial role in organizing data in memory. The reason for emphasizing this lies in the fact that during training, the input layout preference is established. Knowing this preference is crucial during benchmarking and conversion processes, ensuring that the model is configured optimally for compatibility and peak performance during inference.

This stage allows for the integration of extra inputs into your process, depending on the chosen framework and model type. This includes the flexibility to reverse the input channel order from RGB to BGR, specify mean and scale values for each channel in the input image, define input shapes, data types, and values for freezing. Moreover, you can provide the name of the output operation in the model and input the transform value.

For our custom AI model, YOLOv8, once you have chosen the FP16 precision, you do not need to select a data layout format for the PyTorch framework, as it follows the NCHW format. However, you will need to input specific details such as the number of inputs, channels, height, and width during this step. Finally, click “Convert Model” to proceed:

The model will be optimized and converted. You can check on the progress in the status box.

The conversion is completed after a few moments. Detailed information on model conversion commands and logs is available, providing valuable insights for debugging and clarifying the steps taken during the conversion process. A series of “Success” messages are shown in the status box, and you can click Next to continue: