Deep Learning - Crash Course 2023 - ReLU, SoftMax, and Cross Entropy

The video tutorial covers various activation functions used in deep learning, including sigmoid, tanh, ReLU, and softmax. It explains how these functions work, their mathematical representations, and their applications in binary and multiclass classification problems. The tutorial also discusses the use of cross entropy loss for evaluating classification models, emphasizing its effectiveness in handling probability-based outputs.

Which activation function is known for scaling output values between zero and one?

• ReLU

• Sigmoid

• Tanh

• Softmax

What is the primary use of the softmax activation function?

• Binary classification

• Multiclass classification

• Regression analysis

• Data normalization

How does the softmax function transform a vector of real numbers?

• By converting them into binary values

• By applying a linear transformation

• By normalizing them into a probability distribution

• By scaling them between -1 and 1

What is the mathematical operation performed by the softmax function on each element of the input vector?

• Division by the maximum value

• Subtraction of the mean

• Application of the exponential function

• Multiplication by a constant

What is the purpose of using cross-entropy loss in classification problems?

• To increase the model's complexity

• To measure the divergence of predicted probabilities from actual outcomes

• To simplify the model's architecture

• To enhance the model's speed

Which loss function is particularly effective for probability-related outputs?

• Mean Squared Error

• Hinge Loss

• Cross-Entropy Loss

• Huber Loss

What happens to cross-entropy loss as the predicted probability diverges from the actual variable?

• It becomes zero

• It increases

• It remains constant

• It decreases