Current algorithms lean more towards narrow learning, meaning they are good at learning and solving specific types of problems under specific conditions.
Current algorithms lean more towards narrow learning, meaning they are good at learning and solving specific types of problems under specific conditions. These algorithms take a humongous amount of data as compared to humans who can learn from relatively few learning encounters. The transfer process of these learnings from one problem domain to another domain is somewhat limited as well.
Recently, reinforcement learning (RL) has been gaining popularity compared to other deep learning techniques. The buzz around reinforcement learning started with the advent of AlphaGo by DeepMind. AlphaGo was built to play the very complex game of Go. The essence of RL is that it can train models through the interaction with the environment and learn and calibrate from their mistakes. Learning happens through a delayed and cumulative reward system where an agent deduces an action, which then acts on the environment to make a state change. The agent takes the next best action based on the optimized delayed reward. The system retains the learning and recalls the best action when a similar circumstance arises.
This feature of RL — to improve and evolve without constant human or programmatic intervention — makes it interesting to real-world problems like autonomous driving. The autonomous driving puzzle cannot be solved by conventional AI alone, which typically leverages computer vision using Convolutional Neural Networks (CNNs). Autonomous driving cannot be modeled as a supervised learning problem due to strong interaction with the environment, including other vehicles, pedestrians, driver behavior, and road infrastructures. At an abstract level, an autonomous driving agent is an implementation of three steps of sequential tasks: sense (recognize), plan, and control.
Figure 2: Autonomous Driving sequential tasks
The recognition problem has been solved with a high degree of accuracy with advancements in computer vision. We now have the capability to detect pedestrians, curb space, free space between vehicles, traffic signs with low computing power, and high accuracy. Path planning is the most difficult piece of the puzzle. One needs to take a series of environmental inputs and incorporate recognitions and predictions to chart the future driving actions that maneuver the vehicle safely to its destination (reward) by avoiding any accidents/delays (penalties). The control task is relatively easy, as it simply involves passing the signal to either speed (brake, accelerator) or direction control (steering).
What makes RL so attractive and suitable for autonomous driving is the fact that driving is a multi-player, multi-state problem that involves implicit negotiations and interactions. There can literally be thousands of combinations while entering or exiting a freeway ramp or negotiating a crowded roundabout. The driver's temperament, skill level, and experience level cannot be programmed with supervised learning. Through exploration and exploitation techniques, RL can be a great tool for boundary cases, as it can learn from its own experiences and actions that lead to a reward. RL, in a way, closely mimics human decision making — it is like learning to ride a bicycle by trial and error. Mathematically, this state model is best explained with the Markov Decision Process (MDP).
Advancements in reinforcement learning are slowly addressing some of the challenges of huge training data requirements and intense computing needs. There are new advancements in the DQN (Deep Q Network where Q mathematically models the reward function), where an AI agent can learn to drive just by observing the synthetic scenes with virtually simulated miles. The amazing thing is that this learning can happen without much prior information about actual physically driven miles. DQNs currently do have some limitations especially when it comes to dealing with high-dimensional observation space like autonomous driving, which is a continuous domain. Significant progress is being made in this space with Google Deepmind’s innovations with the Deep Deterministic Policy Gradient (DDPG) algorithms to address these limitations.
Foundational concepts in the fields of Machine Learning and Deep Neural Networks. Now we will learn how to use one of the most exciting tools and self-driving car development, deep neural networks. A deep neural network is just a term that describes a big multi-layer neural network.
Introduction to Neural Networks For Self Driving Cars. Foundational concepts in the fields of Machine Learning, Deep Neural Networks and Self Driving Cars
Artificial Intelligence (AI) will and is currently taking over an important role in our lives — not necessarily through intelligent robots.
Introduction to Convolutional Neural Networks for Self Driving Cars. Introductory concepts in the field of Image Recognition using Convolutional Neural Networks
Foundational Concepts in the field of Deep Learning and Machine Learning. We’ll focus on TensorFlow because if one becomes a machine learning expert, these are the tools that people in the trade use everyday.