Autoencoder and Latent Space Quiz

When visualizing the 2D latent space of an Autoencoder (AE) for digit clustering, what would be the expected observation if the AE is learning effectively?

• Digits of the same class are widely scattered across the latent space.

• Digits of different classes overlap significantly in the latent space.

• Digits of the same class form distinct clusters in the latent space.

• The latent space shows no discernible patterns, regardless of digit class.

Which dimensionality reduction technique is commonly used to visualize a high-dimensional latent space in 2D or 3D?

• Principal Component Analysis (PCA)

• t-Distributed Stochastic Neighbor Embedding (t-SNE)

• Both A and B

• Linear Regression

If the latent space of an AE/VAE shows good separation between digit clusters, what does this imply about the encoder's performance?

• The encoder is failing to compress information effectively.

• The encoder is learning meaningful and discriminative features.

• The encoder is overfitting to the training data.

• The encoder is only learning noise.

When coloring the visualized latent space based on labels, what is the primary purpose?

• To make the plot aesthetically pleasing.

• To identify the specific digit each point represents and evaluate clustering quality.

• To increase the dimensionality of the latent space.

• To obscure any patterns that might exist.

What is a potential drawback of using t-SNE for latent space visualization compared to PCA?

• t-SNE always preserves global structure better than PCA.

• t-SNE is a linear dimensionality reduction technique.

• t-SNE's results can be sensitive to hyperparameter choices and may not always preserve global distances.

• t-SNE cannot be used for visualizing latent spaces.

In an Autoencoder, what does the Mean Squared Error (MSE) loss typically measure?

• The difference between the input and the latent vector.

• The pixel-wise squared difference between the original input and its reconstruction.

• The sum of squared differences between encoder and decoder weights.

• The entropy of the latent space distribution.

If you swap MSE with Mean Absolute Error (MAE) in an Autoencoder's reconstruction loss, how might the training dynamics change?

• MAE will always converge faster than MSE.

• MAE is less sensitive to outliers compared to MSE.

• MAE tends to penalize larger errors more severely than MSE.

• MAE requires activation functions that output values between 0 and 1.

When might Binary Crossentropy (BCE) be a more suitable reconstruction loss for an Autoencoder than MSE or MAE?

• When the input images have continuous pixel values (e.g., 0-255).

• When the Autoencoder is used for regression tasks.

• When the input images are binary or normalized to be probabilities (e.g., 0 or 1).

• When the latent space is constrained to follow a Gaussian distribution.

What might be a consequence of using MAE as a reconstruction loss compared to MSE in terms of reconstruction quality?

• MAE typically results in smoother reconstructions with fewer sharp edges.

• MAE might lead to blurrier reconstructions as it averages errors linearly, potentially obscuring fine details.

• MAE always produces higher PSNR values than MSE.

• MAE is more prone to producing noisy reconstructions.

What is the primary reason for plotting learning curves when experimenting with different loss functions in an AE?

• To compare the aesthetic quality of reconstructions visually.

• To track the evolution of loss values over epochs and assess convergence and stability.

• To determine the optimal learning rate automatically.

• To visualize the latent space distribution.