Perceptron & the MLP

What is a Perceptron?

A Perceptron is a basic building block of artificial intelligence inspired by human brain cells.

It takes several numerical inputs, multiplies each input by an assigned weight, adds them together along with a bias number, and decides an output.

Math inside a single neuron:

output = activation( (input1 * weight1) + (input2 * weight2) + bias )

The weights control how important each input is. The bias helps shift the trigger point where the neuron fires.

The Limits of a Single Neuron

A single perceptron can solve simple problems like AND or OR logic gates where points can be separated by drawing a single straight line.

However, in 1969 researchers proved that a single perceptron cannot solve the XOR problem. Points arranged in an XOR pattern need a curved or multi angled boundary. This discovery temporarily slowed AI research until multi layer networks were developed.

Multi Layer Perceptrons

A Multi Layer Perceptron stacks many neurons into connected layers:

1. Input Layer: Receives raw data numbers like age, income, or pixel brightness.
2. Hidden Layers: Intermediate layers where neurons extract patterns.
3. Output Layer: Produces the final prediction, such as a class label or predicted price.

By placing non linear activation functions between hidden layers, the network can bend decision boundaries into complex shapes, allowing it to learn almost any pattern in data.

Say this out loud

A perceptron multiplies inputs by weights, adds a bias, and passes the sum through an activation function. A single perceptron can only draw straight decision lines. Multi Layer Perceptrons stack multiple layers of neurons with non linear activations, allowing the network to learn complex patterns for real world tasks.

Followups to expect

1. What happens if you do not use non linear activation functions between layers? Combining multiple linear layers without non linear activation steps mathematically collapses into a single linear layer, losing the ability to learn complex curves.
2. How are weights updated during training? The network makes a prediction, calculates the loss error against ground truth, and uses backpropagation to adjust weights in the direction that lowers error.