Generative Models and Learning Quiz

What is the main difference between generative models and discriminative models?

• Generative models only describe the conditional distribution of the output for a given input, while discriminative models describe the joint distribution of both inputs and outputs.

• Generative models describe the joint distribution of both inputs and outputs, while discriminative models only describe the conditional distribution of the output for a given input.

• Generative models have a deeper understanding of the data, while discriminative models can simulate synthetic data.

• Generative models are designed to learn from data how to predict the output conditionally on a given input, while discriminative models are learned from data but have a wider scope.

What is the purpose of a generative model in machine learning?

• To predict the output conditionally on a given input.

• To learn from data how to predict the output conditionally on a given input.

• To describe the joint distribution of both inputs and outputs.

• To simulate synthetic data from the model.

What is the Gaussian Mixture Model (GMM) used for in supervised learning?

• Training linear or quadratic discriminant analysis methods.

• Solving the clustering problem.

• Clustering unlabelled data points.

• Learning from partially labelled data.

What is the key insight for using a generative model to make predictions?

• To simulate data from the model.

• To predict the output directly from the input.

• To compute the conditional distribution of the output given the input.

• To learn the joint distribution of the input and output.

What is the purpose of the Expectation-Maximisation (EM) algorithm in semi-supervised learning?

• To discard unlabelled data points.

• To predict missing output values for unlabelled data points.

• To update the model parameters using labelled data only.

• To learn the GMM from labelled data points only.

What is the key difference between the GMM model and k-means clustering?

• GMM is a supervised learning method, while k-means is an unsupervised clustering method

• GMM uses soft cluster assignments, while k-means uses hard cluster assignments

• GMM is sensitive to input normalization, while k-means is not

• GMM uses Euclidean distance, while k-means uses Mahalanobis distance

What is the purpose of applying a clustering algorithm to the inputs in the music classification data Example 2.1?

• To demonstrate the impact of input normalization on clustering results

• To compare the performance of GMM and k-means algorithms

• To illustrate the use of clustering without considering labels

• To find the optimal number of clusters for the data

What is the main challenge in the unsupervised maximum likelihood problem for a GMM?

• The likelihood is unbounded due to degeneracy issues

• The EM algorithm does not converge to a stationary point

• The model parameters are not updated iteratively

• The number of clusters must be specified beforehand

What is the purpose of the EM algorithm in the GMM model?

• To select the clusters that minimise the distances to the cluster centers

• To minimise the sum of pairwise squared Euclidean distances within each cluster

• To perform hard cluster assignments based on cluster probabilities

• To alternate between computing expected log-likelihood and updating model parameters

Why is it important to regularize the GMM model to prevent degeneracy issues?

• To improve the accuracy of cluster assignments

• To avoid unbounded likelihood due to covariance approaching zero

• To speed up the convergence of the EM algorithm

• To increase the flexibility of the model