Components of a Neural Network
Class 12 · Artificial Intelligence
6.3 Components of a Neural Network
A neural network is made up of several interconnected components that work together to process information, learn from data and generate predictions.
Each component has a specific role in determining how information is received, processed and transmitted through the network.
The major components of a neural network are neurons, weights, activation functions, bias, connections, learning rules and propagation functions.
1. Neurons (Nodes)
Neurons, also called nodes, are the fundamental building blocks of a neural network.
A neuron receives inputs from external sources or other neurons, performs a mathematical calculation and produces an output.
A neuron generally calculates a weighted sum of its inputs, adds a bias and then applies an activation function to produce its output.
2. Weights
Weights represent the strength or importance of the connections between neurons.
During training, the neural network learns suitable values for these weights so that it can reduce errors in its predictions.
A higher or lower weight determines how strongly a particular input influences the output of a neuron.
3. Activation Functions
An activation function determines the output of a neuron based on its input.
Activation functions introduce non-linearity into a neural network. This allows the network to learn and represent complex patterns in data.
Common activation functions include:
- Sigmoid
- Tanh
- ReLU
4. Bias
Bias is a constant value added to the weighted sum of inputs before the activation function is applied.
It allows the neural network to shift the activation function and provides additional flexibility when learning patterns from data.
Weighted Sum + Bias → Activation Function → Output
5. Connections (Synapses)
Connections, also referred to as synapses, are the links between neurons.
They allow information to pass from one neuron to another. Each connection has an associated weight that determines the influence of one neuron on another.
6. Learning Rule
A learning rule defines how the neural network adjusts its weights and biases during the training process.
The purpose of learning is to reduce the difference between the predicted output and the expected output.
Backpropagation is an important method used during neural network training to calculate errors and update the network's parameters.
7. Propagation Functions
Propagation functions describe how information moves through a neural network.
There are two important propagation processes:
| Propagation | Purpose |
|---|---|
| Forward Propagation | Input data moves forward through the network from the input layer to the output layer to generate a prediction. |
| Back Propagation | The error is propagated backward through the network so that weights and biases can be adjusted to improve future predictions. |
Components at a Glance
| Component | Role |
|---|---|
| Neurons | Process inputs and produce outputs. |
| Weights | Represent the strength or importance of connections. |
| Activation Functions | Determine neuron output and introduce non-linearity. |
| Bias | Provides an additional constant value to the weighted sum. |
| Connections | Link neurons and allow information to flow between them. |
| Learning Rule | Determines how weights and biases are adjusted during training. |
| Propagation Functions | Define how information and errors move through the network. |
Competency-Based Question
A neural network is being trained to identify whether an image belongs to a particular category. During training, the network changes the numerical values associated with the connections between neurons after comparing its prediction with the actual result.
Identify the component being adjusted and explain its role.
Click to View Answer
The component being adjusted is the weight. Weights represent the strength or importance of connections between neurons. During training, the neural network adjusts these values to reduce prediction errors and improve its performance.
Common Beginner Mistakes
- Confusing a neuron with a connection between neurons.
- Thinking that weights are the same as input values.
- Forgetting that activation functions introduce non-linearity.
- Confusing bias with the weight of an input.
- Thinking that backpropagation produces the initial prediction. Forward propagation produces the prediction.
- Assuming that the learning rule refers only to changing the input data.
- Confusing an activation function such as ReLU or Sigmoid with a machine learning algorithm such as Linear Regression.
Quick Revision
- Neurons: Basic processing units of a neural network.
- Weights: Represent the strength or importance of connections.
- Activation Functions: Determine neuron output and introduce non-linearity.
- Bias: A constant added to the weighted sum before activation.
- Connections: Links through which information passes between neurons.
- Learning Rule: Defines how weights and biases are adjusted during training.
- Forward Propagation: Moves information from input towards output.
- Back Propagation: Uses error information to adjust network parameters.
Memory Trick
Remember the major components using:
N-W-A-B-C-L-P
- N → Neurons
- W → Weights
- A → Activation Functions
- B → Bias
- C → Connections
- L → Learning Rule
- P → Propagation Functions
Exam Tips
- Learn the seven major components: Neurons, Weights, Activation Functions, Bias, Connections, Learning Rule and Propagation Functions.
- For activation functions, remember examples such as Sigmoid, Tanh and ReLU.
- In questions about weights, mention that they represent the strength or importance of connections.
- In questions about bias, remember that it is a constant added to the weighted sum.
- Remember the difference between forward propagation and back propagation.
- For questions asking how a network learns, mention the adjustment of weights and biases.
- Do not identify Linear Regression as an activation function. Sigmoid, Tanh and ReLU are examples of activation functions.
Frequently Asked Questions (FAQs)
1. What are the main components of a neural network?
The main components are neurons, weights, activation functions, bias, connections, learning rules and propagation functions.
2. What are neurons in a neural network?
Neurons or nodes are the basic processing units of a neural network. They receive inputs, process them and produce outputs.
3. What is the purpose of weights?
Weights represent the strength or importance of connections between neurons and are adjusted during training.
4. What is an activation function?
An activation function determines the output of a neuron and introduces non-linearity, allowing the network to learn complex patterns.
5. Name three common activation functions.
Three common activation functions are Sigmoid, Tanh and ReLU.
6. What is bias in a neural network?
Bias is a constant value added to the weighted sum before the activation function is applied.
7. What is a learning rule?
A learning rule defines how the network adjusts its weights and biases during training.
8. What is forward propagation?
Forward propagation is the process in which input data moves through the network from the input layer towards the output layer to generate a prediction.
9. What is back propagation?
Back propagation is the process of moving error information backward through the network to fine-tune weights and biases and improve the network's performance.
Summary
- Neurons or nodes are the fundamental processing units of a neural network.
- Weights determine the strength or importance of connections between neurons.
- Activation functions determine neuron output and introduce non-linearity.
- Bias is a constant added to the weighted sum before the activation function is applied.
- Connections or synapses link neurons and allow information to pass between them.
- The learning rule determines how weights and biases are adjusted during training.
- Forward propagation generates predictions by moving data from input towards output.
- Back propagation uses prediction errors to adjust network parameters and improve future predictions.
Next Topic: Working of a Neural Network