NLP - Lecture 1

Question 1: What is the primary objective of tokenization in Natural Language Processing (NLP)?

• A. To remove stop words
• B. To convert tokens to vectors
• C. To split a document into words, numbers, and punctuation
• D. To perform part-of-speech tagging

Question 2: What is the main difference between stemming and lemmatization in text preprocessing?

• A. Stemming preserves distinctions like adjectives versus nouns
• B. Lemmatization reduces words to their dictionary form, considering word roles
• C. Stemming is more accurate and preferred
• D. Lemmatization only works for English text

Question 3: Why is the removal of stop words not recommended for sentiment analysis?

• A. It makes the text more understandable
• B. It reduces the size of the vocabulary
• C. It changes the meaning of the text
• D. It improves the performance of sentiment analysis

Question 4: What is a common method to represent each token in NLP after tokenization?

• A. Removing the token completely
• B. Converting each token to a vector represented by a Token ID

• C. Transforming tokens into subwords
• D. Combining tokens into longer phrases

Question 5: In NLP, which library provides lemmatization as a preprocessing technique?

• A. spaCy
• B. NLTK
• C. TensorFlow
• D. Keras

Question 6: A Token could be ?

• A word.

• A character

• A piece of word

• A punctuation

• A full sentence

Question 7: When is it advisable to remove stop words during text preprocessing in NLP?

• For topic modeling where context doesn't matter.
  

• For sentiment analysis where context is crucial

• For tasks with constrained memory and processing resources.

• For large-scale keyword search applications.