Data Science and Machine Learning (Theory and Projects) A to Z - Vanishing Gradients in RNN: LSTM

The video discusses the significance of recurrent neural networks (RNNs) and their ability to handle long-term dependencies. It compares Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRU), highlighting their differences in gate structures and parameter usage. The video explains the components of an LSTM unit, including the forget, update, and output gates, and discusses the complexity and flexibility of LSTMs compared to GRUs. It also introduces bidirectional RNNs, which process information from both past and future contexts to improve target prediction.

What was a significant breakthrough in recurrent neural networks that appeared in 1997?

• Support Vector Machines

• Convolutional Neural Networks

• Long Short-Term Memory Networks

• Generative Adversarial Networks

Which gate is unique to LSTMs and not present in GRUs?

• Update Gate

• Forget Gate

• Relevance Gate

• Candidate Gate

In a simplified version of GRU, which gate is primarily used?

• Update Gate

• Forget Gate

• Relevance Gate

• Output Gate

What is the role of the forget gate in an LSTM unit?

• To compute the candidate activation

• To decide whether to update the current cell state

• To determine the next activation

• To decide whether to retain or discard previous memory

Which of the following is a reason why GRUs might be preferred over LSTMs?

• GRUs have an additional gate

• GRUs have more parameters

• GRUs are less flexible

• GRUs are simpler and require fewer parameters

What is the main advantage of bidirectional recurrent neural networks?

• They can process information from both past and future contexts

• They require fewer parameters

• They process information faster

• They are easier to train

Why might it be beneficial to have knowledge of future time steps in RNNs?

• To reduce computational complexity

• To improve the accuracy of predictions

• To simplify the model architecture

• To decrease training time