Top Qs
Timeline
Chat
Perspective
OpenVINO
Toolkit for deploying inference neural network model on Intel hardware From Wikipedia, the free encyclopedia
Remove ads
OpenVINO is an open-source software toolkit for optimizing and deploying deep learning models. It enables programmers to develop scalable and efficient AI solutions with relatively few lines of code. It supports several popular model formats[2] and categories, such as large language models, computer vision, and generative AI.
Actively developed by Intel, it prioritizes high-performance inference on Intel hardware but also supports ARM/ARM64 processors[2] and encourages contributors to add new devices to the portfolio. It sees great use in AI Sound Processing drivers when tied with Intel's Gaussian & Neural Accelerator (GNA).
Based in C++, it offers the following APIs: C/C++, Python, and Node.js (an early preview).
OpenVINO is cross-platform and free for use under Apache License 2.0.[3]
Remove ads
Workflow
The simplest OpenVINO usage involves obtaining a model and running it as is. Yet for the best results, a more complete workflow is suggested:[4]
- obtain a model in one of supported frameworks,
- convert the model to OpenVINO IR using the OpenVINO Converter tool,
- optimize the model, using training-time or post-training options provided by OpenVINO's NNCF.
- execute inference, using OpenVINO Runtime by specifying one of several inference modes.
Remove ads
OpenVINO model format
OpenVINO IR[5] is the default format used to run inference. It is saved as a set of two files, *.bin and *.xml, containing weights and topology, respectively. It is obtained by converting a model from one of the supported frameworks, using the application's API or a dedicated converter.
Models of the supported formats may also be used for inference directly, without prior conversion to OpenVINO IR. Such an approach is more convenient but offers fewer optimization options and lower performance, since the conversion is performed automatically before inference. Some pre-converted models can be found in the Hugging Face repository.[6]
The supported model formats are:[7]
- PyTorch
- TensorFlow
- TensorFlow Lite
- ONNX (including formats that may be serialized to ONNX)
- PaddlePaddle
- JAX/Flax
Remove ads
OS support
See also
References
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads