This blog post explores the principles behind how 360-degree cameras generate images around a vehicle and correct distortion to enhance driver safety.
Various devices exist to assist drivers when parking or navigating narrow roads. Among these, particularly noteworthy is the system that uses footage from cameras mounted around the vehicle to create a bird’s-eye view of the 360° surroundings. This image is then displayed on an in-car monitor for the driver. This device helps drivers safely navigate and park by providing an instant overview of the surrounding environment. Now, let’s examine the process of how this image is presented to the driver.
First, a grid pattern is laid out on the ground around the vehicle, which is then captured by the cameras. The cameras used in this system typically feature wide-angle lenses, providing a large field of view. This reduces blind spots, helping the driver maintain better visibility. However, wide-angle lenses inherently distort images due to the curvature of light passing through the lens. The center of the image appears convex, and the distortion increases as you move away from the center. This is called lens distortion. The characteristics of the camera itself that influence this distortion are called internal variables and are represented by distortion coefficients. Accurate knowledge of the internal variables allows setting a distortion model to correct the distortion.
The process of correcting distortion requires highly sophisticated work. Only by minimizing distortion in the images captured by the camera can the images seen by the driver match the actual situation as closely as possible. Distortion correction algorithms are used for this purpose, and in this process, the characteristics of the lens, along with the position and angle of the camera mounted on the vehicle, play a crucial role. Distortion caused by factors like the tilt of the vehicle-mounted camera is termed an external variable. By comparing the captured image with a real-world grid plate, one can determine the camera’s tilt angle based on the rotation angle of the grid plate in the image or changes in its position. This information is used to modify the external variables and correct the distortion.
Once distortion correction is complete, the next step requires perspective transformation. This involves estimating the corresponding 3D real-world points for the points in the image, thereby obtaining an image with perspective effects removed. Generally, when a camera projects the 3D real world onto a 2D image, objects of the same size appear smaller the farther they are from the camera. However, since an image viewed from a top-down perspective should not show size changes in objects based on distance, removing this perspective effect is crucial.
If we know the positions of several points in the image obtained through viewpoint transformation and their corresponding points on the real-world grid, we can describe the correspondence between all points in the image and the grid points using a virtual coordinate system. Using this correspondence, placing the image points onto a plane while maintaining the grid’s shape and the relative sizes between grids identical to the real world results in a two-dimensional image. This resulting image is precisely the bird’s-eye view image. By synthesizing images from each direction in this manner, the driver can view a 360° image on the monitor, as if looking down from above around the vehicle.
The technology used in this process is highly complex and precise, but the result provides significant assistance to the driver. Particularly in narrow parking spaces or complex road situations, such devices play a crucial role in ensuring driver safety. The advancement of this technology greatly enhances the safety and convenience of vehicle operation and will serve as a vital foundational technology for the development of future autonomous vehicles.
