#OnDeviceAI #TELROP #jetson #nvidia #agx #orin #ai #block #optimize #pipeline #performance #tops #dla #vpi
If you are attempting to deploy Deep Learning / AI models on end-point devices, the NVIDIA Jetson platform is an excellent choice. Indeed, many companies currently use the Jetson series in robots, autonomous driving systems, and data collection devices.
Below is the block diagram of the AGX Orin SoC. While the internal structure differs for each model in the Jetson series, various dedicated hardware blocks are available. While the performance of CPU or GPU blocks is paramount, the section we want to focus on here is the ACCEL (DLA, PVA, ISP, DECODE, ENCODE) block.
Looking at the table below, you can see that the Jetson series is divided into various grades based on performance, architecture, and IO types. Among these, the most critical factor is AI Performance, specifically the TOPs (Tera Operations Per Second) value. As shown in the table, the AGX Orin 64GB module achieves 275 TOPs in the case of INT8.
However, the table below reveals how this figure of 275 TOPs is constituted. The GPU alone only provides 171 TOPs; the full 275 TOPs performance is achieved only when the DLA (Deep Learning Accelerator) blocks are utilized together. Furthermore, using DLA is significantly more advantageous in terms of energy efficiency compared to using the GPU.
In addition to DLA blocks, using VPI (Vision Programming Interface), which utilizes blocks like PVA, VIC, and OFA, demonstrates superior performance compared to simply using the CPU/GPU. In other words, finding the optimal operating environment requires understanding hardware characteristics and configuring the deep learning pipeline according to the situation.
Considering the above, leveraging the outstanding AI performance and TOPs values of the NVIDIA Jetson series is essential in robotics environments that require diverse and rapid processing. Utilizing hardware accelerators like DLA maximizes this performance, and solutions like TELROP Cloud enable the construction of systems optimized for user requirements.
Vision Algorithm & AI Model NVIDIA On-Device AI Optimization
DeepStream Camera Data Pipeline Optimization
AI Inferencing, Encoding, and Streaming Data Pipeline Optimization for Cameras
DLA (Deep Learning Accelerator)
VPI (Vision Programming Interface)
Optimization of vision-dedicated Hardware Accelerators such as PVA, VIC, and OFA, in addition to CPU and GPU.
Power Usage Analysis & Optimization
TELROP Cloud-Based Remote Operation Solution
Robot System Simplification
Support for interface customization required for robots, including Lidar, radar, camera, CAN, GPS, and IMU.
High-Quality Vision System Construction
Optimal System Application by Use Case
sales@telelian.com
#OnDeviceAI #TELROP #jetson #nvidia #agx #orin #ai #block #optimize #pipeline #performance #tops #dla #vpi
If you are attempting to deploy Deep Learning / AI models on end-point devices, the NVIDIA Jetson platform is an excellent choice. Indeed, many companies currently use the Jetson series in robots, autonomous driving systems, and data collection devices.
Below is the block diagram of the AGX Orin SoC. While the internal structure differs for each model in the Jetson series, various dedicated hardware blocks are available. While the performance of CPU or GPU blocks is paramount, the section we want to focus on here is the ACCEL (DLA, PVA, ISP, DECODE, ENCODE) block.
Looking at the table below, you can see that the Jetson series is divided into various grades based on performance, architecture, and IO types. Among these, the most critical factor is AI Performance, specifically the TOPs (Tera Operations Per Second) value. As shown in the table, the AGX Orin 64GB module achieves 275 TOPs in the case of INT8.
However, the table below reveals how this figure of 275 TOPs is constituted. The GPU alone only provides 171 TOPs; the full 275 TOPs performance is achieved only when the DLA (Deep Learning Accelerator) blocks are utilized together. Furthermore, using DLA is significantly more advantageous in terms of energy efficiency compared to using the GPU.
In addition to DLA blocks, using VPI (Vision Programming Interface), which utilizes blocks like PVA, VIC, and OFA, demonstrates superior performance compared to simply using the CPU/GPU. In other words, finding the optimal operating environment requires understanding hardware characteristics and configuring the deep learning pipeline according to the situation.
Considering the above, leveraging the outstanding AI performance and TOPs values of the NVIDIA Jetson series is essential in robotics environments that require diverse and rapid processing. Utilizing hardware accelerators like DLA maximizes this performance, and solutions like TELROP Cloud enable the construction of systems optimized for user requirements.
Vision Algorithm & AI Model NVIDIA On-Device AI Optimization
DeepStream Camera Data Pipeline Optimization
AI Inferencing, Encoding, and Streaming Data Pipeline Optimization for Cameras
DLA (Deep Learning Accelerator)
Application of effective AI algorithms that are more power-efficient than GPUs and capable of parallel execution for AI vision models.
VPI (Vision Programming Interface)
Optimization of vision-dedicated Hardware Accelerators such as PVA, VIC, and OFA, in addition to CPU and GPU.
Power Usage Analysis & Optimization
Analysis of hardware usage based on AI algorithm structure and power efficiency optimization (Can reduce GPU usage by over 90%).
TELROP Cloud-Based Remote Operation Solution
Robot System Simplification
Support for interface customization required for robots, including Lidar, radar, camera, CAN, GPS, and IMU.
High-Quality Vision System Construction
Support for high-quality AI perception camera modules, GMSL2 interface, and the latest H.265/AV1 codecs.
Optimal System Application by Use Case
Efficient application tailored to specific purposes through various performance Edge AI devices based on NVIDIA (20~275 TOPs).
sales@telelian.com