Data Science and Machine Learning (Theory and Projects) A to Z - Vanishing Gradients in RNN: Attention Model Optional

The video tutorial explains the attention mechanism in machine translation, focusing on the mathematical details and workings of bidirectional networks. It covers the flow of activations in these networks and the role of the decoder network in producing translations. The tutorial also delves into the technical aspects of weight calculations in the attention model, including constraints and the use of softmax for normalization.

What is the primary focus of the attention mechanism discussed in the video?

• Data compression

• Speech synthesis

• Machine translation

• Image recognition

In a bidirectional network, how are activations typically combined?

• By summing them

• By concatenating them

• By multiplying them

• By averaging them

What is the role of the decoder network in the context of machine translation?

• To encode the input data

• To produce translations

• To generate random outputs

• To compress the data

How are the weights in the attention mechanism constrained?

• They must be negative

• They must be non-negative and sum to one

• They must sum to zero

• They must be greater than one

What mathematical function is used to ensure the weights are properly constrained?

• Softmax

• Sigmoid

• ReLU

• Tanh

What is the purpose of using a weighted average in the attention mechanism?

• To simplify the calculations

• To blend activations based on importance

• To increase the complexity

• To ensure equal contribution of all inputs

Which of the following is NOT a constraint on the alpha values in the attention mechanism?

• They must be less than one

• They must be greater than one

• They must sum to one

• They must be non-negative