Edge Developer Toolbox Developer Guide
Import AI model from Hugging Face
AI models published on Hugging Face* for community consumption can now be imported from within the Deep learning Workbench for optimization.
In this example, let’s import a transformer-based model for benchmarking.
From the Deep Learning Model Explorer and Workbench, select Import from Hugging Face.
Select a model. You can choose from many Generative AI models in the Intermediate Representation (IR) format used by the OpenVINO™ toolkit. As an example, we will select the llama-3-8B-Instruct-ov-fp16 text generation model. Once a model is chosen, click Import.
Select the llama-3-8B-Instruct-ov-fp16 text generation model for this example.
After the model is successfully imported, click Next.
Provide a prompt and click Next.
If importing a model from Hugging Face or importing a custom model with OpenVINO framework, skip to the Select Target Environment and Benchmarking section.
If importing a model from Open Model Zoo or a custom model not using the OpenVINO framework, proceed to the Convert and Optimize Model with OpenVINO™ Toolkit section.