Generative AI: Input and Pretraining

What are the four key steps that happen inside a chatbot after you hit enter?

• Input, storage, processing, output

•  Input, analysis, generation, response

• Question, thinking, answering, learning

• Tokenization, embedding, decoding, output

 Before a language model can be used, what process does it undergo?

• Fine-tuning

• Pre-training

• Optimization

• Calibration

What analogy is used to describe the pre-training process of a language model?

• Like a model going to the gym

• Like a model going to school

• Like a model reading books

• Like a model practicing skills

What type of data does the model study during pre-training?

• Only academic textbooks

• Primarily scientific papers

• Almost every web page on the internet

• Exclusively news articles

What are three things the model learns during pre-training?

• Recognize words, grasp meanings, understand language rules

• Solve math problems, write essays, answer questions

• Create images, compose music, translate languages

• Predict weather, analyze data, generate code

What does the chatbot do with the user's question in the first step?

• Translates it into multiple languages

• Searches the internet for answers

• Breaks it down into bite-sized pieces

• Analyzes the sentiment of the question

What are the smaller parts of words called when the chatbot breaks down the input?

• Bits

• Bytes

• Tokens

• Fragments

What analogy is used to describe the process of breaking up the input into individual pieces?

• Like solving a puzzle

• Like noticing writing or signs we should pay attention to

• Like organizing a library

• Like sorting through a messy room

What does the model do with symbols it doesn't recognize during the input processing?

• Tries to learn their meaning

• Replaces them with similar known symbols

• Ignores them

• Asks the user for clarification