How-important-is-deep-learning-in-autonomous-driving

Deep learning (DL) is a very interesting technology indeed and yes it does solve perception really well however I believe it’s not currently good enough for autonomous driving cars. Autonomous cars are like 10 - 20 yrs away from now. DL has some very interesting properties such as being able to automatically learn complex mapping functions and the ability to scale up. Such properties are important in many real-world applications such as large-scale image classification and recognition.

Most machine learning (ML) algorithms performance plateaus at a certain point while DL algorithms easily scale up to very large scales and hence can make good use of big data. DL is probably the only ML algorithm that is able to leverage the huge amounts of training data that comes from autonomous car sensors such as the camera system.

DL can be used in self-driving cars to process sensory data and make informed decisions. Just to expand a bit on Roman Trusov answer, DL can indeed be used for:

Lane detection: This is useful for proper driving, as the car needs to know on which side of the road it is. Lane detection also makes it easy to follow along the curving road in many conditions.
Pedestrian detection: The system must detect presence of humans in a scene as it drives. The system needs to know whether an object is a pedestrian or not so that it can put more emphasis on not hitting pedestrians, that is, it needs to be more careful driving around pedestrians than other less important objects.
Road sign recognition: The system needs to recognize road signs and be able to behave accordingly.
Traffic light detection: The car needs to detect and recognize traffic lights so that it can be compliant with road rules just like human drivers.
Face detection/recognition: Yes a self-driving car needs to detect and recognize the face of the driver or other people inside and maybe also those outside. If the car is connected to some network it can be able to match those faces against a database in order to recognize fugitives or dangerous criminals that may try to use it. Face detection/recognition can also be useful for owner recognition, so that the car can identify its owner just like a pet, how sweet :).
Car detection: It also needs to detect the presence of other cars in the environment.
Obstacle detection: Obstacles can be detected using other means such as using ultrasound but the car needs to also use it’s camera systems to determine presence of obstacles.
Environment recognition: It is important for the system to recognize where it is just by using the camera feed.
Human action recognition: Such as how to interact with other drivers on the road since autonomous cars will drive along side humans for many years to come.
The list goes on, DL systems are very powerful tools indeed but there are some properties that may affect their practicality especially when it comes to autonomous cars.

The two major concerns I can give are:

Unpredictability
Easy to fool
A lot of ML algorithms are actually very unpredictable, yes humans are as well, but the unpredictability of DL systems is worse than that of humans and hence this makes it somehow unsafe to apply DL systems, as it is now, to real-world self-driving cars. The DL system can really guess poorly sometimes especially if the conditions are quite novel. A human on the other hand can use several methods in order to make proper decisions in unforeseen circumstances. That’s why pilots are still flying planes, computers or ML systems are currently not very robust to be left with the task of moving or flying people from point A to point B, they are just not reliable enough yet.

Now when it comes to getting easily fooled, here is a paper on the serious downside of DL systems. Adversarial images, to be precise, are a danger to DL systems, they are a security concern not just for autonomous cars but also for other applications of DL such as in medical imaging and face recognition security systems. There is a theory that the deep learning system learns highly discontinuous mapping functions instead of continuous ones. These discontinuous mapping functions can be very sensitive to subtle perturbations, a human wouldn’t even detect those subtle perturbations. This can be very dangerous if an autonomous car makes a potentially fatal move because of this adversarial effect.

Thus DL is interesting and yes very useful and powerful but it’s not yet mature enough to be given the responsibility of driving cars. Thus for now DL, like most of the autonomous car research, is limited to the experimental phase. I believe more advancements need to be made before DL or other future algorithms can move from the labs to the real-world.

Deep learning as it is now, is not a safe solution to autonomous driving cars.