Neural Networks and Deep Learning
Abstract
Neural networks were developed to simulate the human nervous system for machine
learning tasks by treating the computational units in a learning model in a manner similar
to human neurons. The grand vision of neural networks is to create artificial intelligence
by building machines whose architecture simulates the computations in the human nervous
system. This is obviously not a simple task because the computational power of the
fastest computer today is a minuscule fraction of the computational power of a human
brain. Neural networks were developed soon after the advent of computers in the fifties and
sixties. Rosenblatt’s perceptron algorithm was seen as a fundamental cornerstone of neural
networks, which caused an initial excitement about the prospects of artificial intelligence.
However, after the initial euphoria, there was a period of disappointment in which the data
hungry and computationally intensive nature of neural networks was seen as an impediment
to their usability. Eventually, at the turn of the century, greater data availability and increasing
computational power lead to increased successes of neural networks, and this area
was reborn under the new label of “deep learning.” Although we are still far from the day
that artificial intelligence (AI) is close to human performance, there are specific domains
like image recognition, self-driving cars, and game playing, where AI has matched or exceeded
human performance. It is also hard to predict what AI might be able to do in the
future. For example, few computer vision experts would have thought two decades ago that
any automated system could ever perform an intuitive task like categorizing an image more
accurately than a human.