Data Science and Machine Learning (Theory and Projects) A to Z - DNN and Deep Learning Basics: DNN Activation Functions

The video tutorial introduces activation functions in the Torch library, focusing on sigmoid and ReLU functions. It explains how to define and use these functions with examples, and discusses the possibility of creating custom activation functions. The tutorial also briefly introduces loss functions, setting the stage for understanding their role in training neural networks using gradient descent.

What is the primary purpose of an activation function in a neural network?

• To introduce non-linearity into the model

• To normalize the input data

• To initialize the weights of the network

• To reduce the dimensionality of the data

Which activation function is used in the example to demonstrate its behavior with a negative input value?

• Softmax

• Tanh

• ReLU

• Sigmoid

How does the ReLU activation function behave when the input is negative?

• It outputs the same negative value

• It outputs a positive value

• It outputs zero

• It outputs a value between 0 and 1

What is a key difference between the sigmoid and ReLU activation functions?

• ReLU is computationally more expensive than sigmoid

• ReLU is used for binary classification, while sigmoid is not

• Sigmoid outputs only positive values, while ReLU can output negative values

• Sigmoid can output values between 0 and 1, while ReLU outputs zero or positive values

Why is understanding loss functions important in training neural networks?

• They are used to initialize the model parameters

• They measure how well the model is performing

• They determine the learning rate of the model

• They help in visualizing the data