[Tech]Advantages of Camera ISP DeWarp in Multi-Camera Environments

With advancements in AI, Large Language Models (LLM), and reinforcement learning, there is an ongoing effort to replace high-cost sensors with comparatively easy-to-apply and cost-effective cameras. In autonomous vehicles, one can observe cameras with various fields of view (FOV) not only at the front and rear but also on the sides. Service robots also utilize a significant number of cameras. A crucial aspect of applying vision recognition through cameras in these applications is the correction of camera lens distortion with DeWarp technology.

What is DeWarp?

DeWarp refers to the technology that corrects lens distortion in images captured by cameras. It is especially useful for correcting image deformations that occur with wide-angle or fisheye lenses, providing more accurate and natural visual information. This technology is used to improve recognition rates in various fields such as robotic systems, security cameras, and automotive camera systems. The diagram below shows the degree of lens distortion plotted on a coordinate system.

Using DeWarp on the NVIDIA Jetson Platform

The NVIDIA Jetson is an embedded platform designed for high-performance computing requirements of AI applications. By implementing the DeWarp feature, it is possible to remove distortions from high-resolution video streams in real-time. This optimization can be achieved through the proper use of DeepStream and NVIDIA hardware blocks.

Implementing DeWarp in Camera ISP

As the technology of Image Signal Processors (ISP) in cameras advances, methods to implement DeWarp within the camera's ISP are provided, significantly enhancing processing speed and efficiency.

The ISP processes raw image data received from the camera sensor and converts it into a digital image. By directly handling DeWarp within the ISP, image distortions can be corrected without the need for additional external processing. This reduces the system's power consumption and conserves the main processor's processing capacity, thereby providing opportunities to create more varied applications.

The diagram below represents the results before and after DeWarp processing within the camera ISP.

The Importance of Load Distribution in a Multi-Camera Environment and Camera ISP DeWarp in Robotics

Using multiple cameras in robotics is essential to maximize environmental awareness. Processing images from all these cameras and executing DeWarp on a single processor can lead to significant load, consuming substantial GPU/CPU resources. If the camera's ISP can handle the DeWarp function, it can effectively distribute the load on the central processing unit.

If distortion is processed by the ISP, more GPU/CPU processing power can be allocated to other important tasks, contributing to the enhancement of various robot functions. Implementing DeWarp through the ISP acts as a key factor in improving the overall performance and cost-efficiency of robots.

Implementing DeWarp within the ISP is a critical technical approach that enables robots to operate more effectively in complex environments. This not only benefits robotics but also extends the potential applications in various fields such as automotive and security systems, further broadening the scope of possibilities.

#camera #ADAS #AI #dewarp #GMSL2 #ISP #distortion #deform #geosemi #jetson #nvidia


B207, IT College, 1342, Seongnam-daero, Sujeong-gu, Seongnam-si, Gyeonggi-do, Republic of Korea

Copyright ⓒ Telelian Inc.