Deep Learning - Artificial Neural Networks with Tensorflow - Binary Cross Entropy

The lecture explains the cross entropy loss function used in binary classification, emphasizing its basis in probability. It discusses the Bernoulli distribution for binary events, using a coin toss as an example to illustrate maximum likelihood estimation. The lecture details the process of calculating likelihood and log likelihood, highlighting the similarities between binary cross entropy and negative log likelihood. It concludes by comparing the binary cross entropy with mean squared error, noting that both are derived from probability distributions and involve maximum likelihood solutions.

What is the correct loss function to use for binary classification?

• Mean Squared Error

• Cross Entropy Loss

• Huber Loss

• Hinge Loss

Which distribution is typically used for binary events?

• Exponential Distribution

• Bernoulli Distribution

• Poisson Distribution

• Normal Distribution

In the context of a coin toss, what does the Bernoulli PMF calculate?

• Cumulative probability

• Probability density

• Expected value

• Probability mass

What is the first step in maximizing the likelihood function?

• Solving for the mean

• Calculating the log likelihood

• Finding the derivative

• Setting the likelihood to zero

How is the binary cross entropy related to the log likelihood?

• It is the sum of the log likelihood

• It is the derivative of the log likelihood

• It is the negative log likelihood

• It is unrelated to the log likelihood

What is the common pattern between binary cross entropy and mean squared error?

• Both are based on the Bernoulli distribution

• Both are derived from the Gaussian distribution

• Both are forms of negative log likelihood

• Both are used for binary classification

Why do we divide the sum of errors by N in binary cross entropy?

• To decrease the error value

• To make the error value invariant to sample size

• To make the error value dependent on sample size

• To increase the error value