Neural Networks

What is a neuron?

• 1. A model that can process images

• Function with inputs, weights, and an activation function which generates an output

• Neurons are NOT part of neural networks

What is an activation function? [2 CORRECT]

• A function that decides whether a neuron is activated or not

• A function that determines whether or not the feature the neuron detects should have an influence over the end prediction

• A function that activates a motor

• A function that tells you how many neurons to use in a layer

1. A neuron is similar to a logistic regression model. (It uses inputs, weights, and an activation function)

• True

• False

What are the three layer types we covered today?

• input, activation, output

• input, gradient, output

• input, weights, output

• input, hidden, output

Read the image!! ... the answer is TRUE, this question is for review :)

• True

• False

What is forward propagation? [2 CORRECT ANSWERS]

• Gradient descent

• Gradient ascent

• Getting predicted value from our inputs!

• Apply weights, biases, and activation function to inputs to get an output, without tweaking weights and biases!

What is backpropagation?

• Calculates the gradient from the output to the first hidden layer

• 1. Feeding inputs forward into the network to get a predicted value

• 1. Uses gradient descent to find the best learning rate

• 1. Backwards walking through a slope

1. What is the vanishing gradient problem? [2 CORRECT]

• Sigmoid activation function, as well as too many layers can cause it

• Gradient gets way too big

• Gradient becomes negative infinity

• Gradient becomes too close to 0, so gradient descent stops updating weights and biases