LiDAR is a sensor currently changing the world. It is integrated in self-driving vehicles, autonomous drones, robots, satellites, rockets, and many more.
This sensor understands the world in 3D using laser beams and measuring the time it takes for the laser to come back. The output is a point cloud.
It’s a technology harder to understand than the camera because less people have access to it. I had access to it and today will help you understand how LiDAR detection works for self-driving cars.
LiDAR’s underlying technology could power a wide array of use cases, but currently, self-driving vehicles offer us the best method of exploring this technology.
Today, I’ll provide an introduction to LiDAR and explain how it works from the self-driving car perspective. I will then explore how to process the point cloud, detect obstacles using 3D bounding boxes, and segment the drivable area in real-time.
The target goal is something similar to this picture.
A LiDAR sensor works with light. LiDAR stands for Light Detection And Ranging. They can detect obstacles up to 300 meters and accurately estimate their positions. In Self-Driving Cars, this is the most accurate sensor for position estimation.
64–32–16 Layers LiDARs
A LiDAR sensor is composed of two parts: laser emission (top) and **laser reception **(bottom). The emission system works by leveraging layers of laser beams. The more layers, the more accurate the LiDAR will be. On the image above, you can see that the more layers, the bigger the sensor.
When these lasers hit an obstacle, they create a set of point clouds. The sensor works with Time of Flight (TOF). Essentially, it measures the time it takes for every laser beam to reflect and come back.
(Image taken from my self-driving car course)
When at very high quality (and price), LiDARs can create rich 3D scenes of an environment. They can emit up to 2 million points every second.
Point clouds represent the world in 3D. The LiDAR sensor gets the exact (X,Y,Z) position of every impact point.
In the first case, it will focus its detection on a position and offer a coverage range (90° for example). In the latter case, it will rotate around itself and offer a 360° view. In this case, we want to place it on the roof for better visibility.
**LiDARs are rarely used as standalone sensors. They’re often coupled with a camera or a RADAR, in a process called Sensor Fusion. **Please find my Sensor Fusion article here to learn more.
The fusion process can be of two types: Early Fusion or **Late Fusion. **In the first case, the point cloud is fused with the image pixels. In the second case, the individual detections are fused together.
In this article, I will cover the obstacle detection process (late fusion) that generally happens in 4 steps:
The future of machine learning is on the edge. Subscribe to the Fritz AI Newsletter to discover the possibilities and benefits of embedding ML models inside mobile apps.
To process point clouds, we can use the most popular library, called PCL (Point Cloud Library). It’s available in Python, but it makes more sense to use it in C++, as the language is more suited to robotics. It’s also compliant with ROS (Robotic OS).
The PCL library can do most of the computations necessary to detect obstacles, from loading the points to executing algorithms. This library is the computer vision equivalent to OpenCV.
Since the output of LiDAR can easily be 100,000 points per second, we need to use something called a voxel grid to downsample the results.
#machine-learning #lidar #tech #heartbeat #self-driving-cars #deep learning