Introduction to NVIDIA VPI

Key Takeaways

vpi is a software library that is used to create high performance computer vision and image processing pipelines it exposes several computing accelerators available in the device it's running on through a uniform and easy to use interface moreover vpi is currently the only way to have direct access to some of jetson accelerators like the pva used for computer vision algorithms and vic most commonly used for image operations vpi works on tekker devices like the jetson azx xavier shown here and linux pcs on a pc it makes use of the machine cpu and installed nvidia's gpu for acceleration vpi is deployed with nvidia jetpack you can consult jetpack's documentation for instructions on how to install it as an example we show here a complete stereo disparity estimation pipeline that efficiently uses several backends which is how we call the computer accelerators this pipeline receives the input from a stereo camera which are the left and right images of a stereo pair then the backend works on this input to correct lens distortion and scale the image down resulting in a rectified serra pair then images get converted from color to grayscale using the gpu with the result fed into a sequence of operations using pva and nv anc backends the output is an estimate of the disparity between the input images which is related to the scene depth vpi comes with several algorithms ranging from image processing building blocks like the build the box filtering convolution image rescaling and remap to more complex computer vision algorithms like the harris corner detection klt feature tracker optical flow background subtraction among others as it can be seen in the table each algorithm is implemented in one or more backends each implementation yields similar results which allows the user to pick and choose the best backend for execution of an algorithm given the expected workload on each backend the number of algorithms keep on growing based on user feedback on release 1.1 we have included five new algorithms and one more backend for the stereo disparity estimation if you're interested in more information about vpi programming refer to its documentation for an in-depth tutorial you can watch the webinar we presented back in february 2021 entitled implementing computer vision and image processing solutions with vpi vpi is also about speed this chart shows how its cuda and cpu implementations compare to the corresponding opencv algorithms it's an apple so apple's comparison cpu compared to spew cuda compared to cuda separable convolution for instance get a whopping 50 times faster execution than opencv on cpu whereas harris corners is almost 20 times faster than opencv on cuda most other algorithms are not left behind the speed vpi achieves on cpu makes it as a good choice for execution back-end especially when other backends are being fully utilized this allows for better load balancing depending on the pipeline being executed you