Understanding Neural Networks

An artificial neural network (often referred to simply as a neural network) is a computer system modeled after the structure of a biological brain that facilitates machine learning and deep learning.

The human brain is composed of about 100 billion neurons that communicate with one another electrochemically across minute gaps called synapses. A single neuron can have up to 10,000 connections with other neurons. Working together, neurons are responsible for receiving sensory input from the external world, regulating bodily functions, controlling muscle movement, forming and recording memories and thoughts, and more.

Neurons increase the strength of their connections based on learning and practice. Whether you're studying a new language, learning to play a musical instrument, or training for the World Cup, your neurons strengthen existing connections and create new connections for developing the requisite knowledge and skills. That's why the more you practice the better you get. These selected neurons build new and more efficient paths between and among one another. Eventually, with enough study and practice, you perform certain tasks with little to no effort.

How an Artificial Neural Network Is Structured

Instead of being made up of neurons, an artificial neural network consists of nodes. Each node receives input from one or more other nodes or from an external source and computes an output. A node's output is then sent to one or more other nodes in the neural network or is communicated to the outside world. This communication might be as the answer to a question or as the solution to a problem.

Nodes are arranged in layers: an input layer, hidden layers, and an output layer. Data (such as a spoken word or phrase, an image, or a question) enters the input layer, is processed in the hidden layers, and the result is delivered via the output layer.

The Marching Band Analogy: How neural networks work

Picture nodes in a neural network as players in a marching band and each row of band members as a layer. Assume that none of the players knows the music to be played or how to move during the performance. Only the front row of band members can see the band leader (the drum major). The drum major gives the first row a signal that's passed through the remaining rows (layers), enabling all players to coordinate their movements and the playing of their instruments.

At first, players would be bumping into one another and playing the wrong notes, but with more and more practice, the players would get in sync and perform as a unit. They would learn.

To smooth the learning curve, the band creates a system that enables band members to provide feedback using machine learning methods. As they move and play, the band members choose numbers that indicate their level of confidence (say from 0 to 100 percent) that they are doing it right. Based on each band member's confidence level, neighboring band members make small adjustments and then check to see whether their adjustments increased or decreased their neighbor's confidence level. The goal is to achieve a 100 percent confidence level for all band members.

The idea here is that this neural marching band network will learn on its own without additional input or correction from an outside source. Theoretically, at least, the nodes will eventually make enough small adjustments to produce the correct output (a stellar performance) through trial and error, learning from their mistakes using reinforcement learning.

Improving the Learning Process

As you can imagine, learning by trial and error can be very chaotic and time-consuming, especially when you have multiple entities making their own adjustments based on input from numerous other entities. In the case of our fictional marching band, band members would be bumping into one another and playing the wrong notes for hours, days, or weeks before they actually coordinated their efforts.

To overcome this challenge, artificial intelligence developers attempt to tweak the network to make it more efficient. For example, suppose you gave more weight to feedback from the drummers because they set the rhythm. Perhaps you give their confidence level four times the importance as other band members. Now, when the band members make adjustments, they look more to the drummers to determine the net impact of the adjustments they made, and the marching band learns much faster.

Eventually, the band delivers a nicely choreographed and well-orchestrated performance to the output layer. If this were a neural network, the output could then be stored, and whenever instructed to do so, it could repeat its performance. In addition, the strengthened connections between certain neurons might make learning new musical arrangements easier.

Frequently Asked Questions

What is the history of neural networks?

The history of neural networks dates back to the 1940s when Warren McCulloch and Walter Pitts introduced the concept of an artificial neuron.

This idea evolved significantly in the subsequent decades, with notable milestones including the development of the perceptron in the 1950s, the invention of backpropagation in the 1980s, and the advent of deep learning algorithms in the 2000s.

What are the types of neural networks?

There are several types of neural networks, each suited for different tasks.

Major types include feedforward neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and deep neural networks (DNNs).

These networks differ in their architecture and the specific way they process and learn from data.

How is deep learning different from traditional machine learning?

Deep learning is a subset of machine learning that uses deep neural networks to analyze data and make decisions.

Unlike traditional machine learning algorithms that rely on feature extraction, deep learning algorithms automatically extract and learn features from raw data, making them more powerful for tasks like image and speech recognition.

What are the common applications of neural networks?

Neural networks are used in various applications including image and speech recognition, natural language processing, game playing, medical diagnosis, and more.

They are very good at learning complex patterns from data. This makes them highly effective in many fields, which shows the power of deep learning and neural networks.

How are neural networks used in unsupervised learning?

Neural networks can be used for unsupervised learning, where the network tries to learn patterns or features from unlabeled data.

Examples of neural networks used for unsupervised learning include autoencoders and generative adversarial networks (GANs).

What is the role of an artificial neuron in a neural network?

An artificial neuron is the basic unit of a neural network, modeled after the biological neuron.

It receives weighted input signals, applies an activation function to the sum of these inputs, and produces an output.

These neurons are organized into layers, and their connections form the neural network architecture.

Why is generative AI important in the field of neural networks?

Generative artificial intelligence is important because it enables the creation of new data instances that resemble the training data.

Techniques such as generative adversarial networks (GANs) and variational autoencoders (VAEs) have revolutionized applications like image generation, data augmentation, and creating realistic simulations, thereby expanding the scope of what can be achieved with neural networks.

This is my weekly newsletter that I call The Deep End because I want to go deeper than results you’ll see from searches or LLMs. Each week I’ll go deep to explain a topic that’s relevant to people who work with technology. I’ll be posting about artificial intelligence, data science, and data ethics.

This newsletter is 100% human written 💪 (* aside from a quick run through grammar and spell check).