Transformer Quiz

What is the main problem with Recurrent Neural Networks (RNN)?

• Fast computation for long sequences

• No problems at all

• Difficulty in accessing information from long time ago

• No issues with gradients

What is the purpose of positional encoding in the Transformer model?

• To increase the model size

• To add noise to the data

• To represent the position of words in a sentence

• To remove unnecessary words

What is the function of Self-Attention in the Transformer model?

• To add random words

• To remove all words except the most important one

• To ignore certain words

• To relate words to each other

What is the purpose of layer normalization in the Transformer model?

• To speed up computation

• To make the model causal

• To introduce fluctuations in the data

• To remove all layers except the last one

What is the goal of Masked Multi-Head Attention in the Transformer model?

• To make the model causal

• To remove attention from the model

• To focus on future words only

• To ignore the first word

Which task is typically associated with an Encoder Only Transformer?

• Machine translation

• Text summarization

• Sentimental analysis

• Text generation

Which task is typically associated with a Decoder Only Transformer?

• Chatbots

• Question-answering

• Text classification

• Anomaly detection

Which task is typically associated with an Encoder-decoder Transformer?

• Sentimental analysis

• Anomaly detection

• Text summarization

• Text generation

What is the purpose of Beam Search strategy in the Transformer model?

• To select the word with the maximum softmax value

• To remove all words except the most probable one

• To speed up the training process

• To evaluate all possible next words at each step

Why are trigonometric functions like cos and sin used in the Transformer model?

• To remove all patterns

• To confuse the model

• To represent a pattern that the model can recognize as continuous

• To slow down computation