Which choice best describes the distinctions among the different training stages: Pre-training , Supervised Fine-Tuning (SFT) , and Reinforcement Learning from Human Feedback (RLHF) ?

Pre-training uses a large unlabeled text corpus, SFT fine-tunes on labeled examples, and RLHF optimizes model behavior using human preference signals.

Why is pre-training computationally expensive?

The models have billions of parameters, and they process trillions of tokens of data

Scaling laws suggest that model performance improves primarily based on which factors? Options: Learning rate, batch size, and number of training epochs; Inference speed, hardware type (GPU/TPU), and model quantization level; Model Size, dataset size, and compute; Data quality, network architecture type, and optimizer algorithm

Model Size, dataset size, and compute

In the context of large language models, what best describes a "base model"?

A model trained on large amounts of general data without being specialized for any particular task.

A newly developed Large Language Model (LLM) claims a context window of 16,384 tokens. You want the LLM to analyze a 20,000-token legal document all at once. Which of the following statements best describes how the context window limit impacts the model’s ability to handle the document?

The LLM will process a maximum of 16,384 tokens at once; any tokens beyond this limit must either be excluded, truncated, or segmented.

Which of the following statements best describes parallelization in the context of training large language models? Options: It automates hyperparameter selection for faster convergence without human intervention., It splits the training process across multiple processors or devices to reduce overall training time., It compresses the model parameters so the entire model can fit into a single GPU’s memory., It guarantees higher accuracy by forcing each processor to learn a separate part of the model independently.

It splits the training process across multiple processors or devices to reduce overall training time.

What is the main reason a pretrained model cannot immediately be used as a ChatGPT-like chatbot?

It is trained to predict the next likely token, so it will not produce helpful answers.

What is the main limitation of using supervised fine-tuning (SFT) alone for creating an AI assistant?

The model can only learn to imitate examples in the training data, not generalize to new situations.

Why might a model trained with RLHF sometimes "refuse" to answer certain questions that it is technically capable of answering?

The model has learned that certain responses lead to negative rewards even if they're factually correct.

Which of these is typically the smallest dataset? Supervised fine-tuning dataset, Pretraining dataset, Human preference dataset for RLHF, Validation dataset

Human preference dataset for RLHF

What is a common challenge when using RLHF compared to supervised fine-tuning?

RLHF is more likely to have problems with over-optimization.

What is the role of a reward model in RLHF?

To predict which responses humans would prefer

Which statement best describes why evaluation frameworks and benchmarks are critical in post-training alignment for Large Language Models? Options: A. They replace human oversight by automatically correcting misaligned outputs. B. They provide systematic ways to measure performance and safety, identifying where the model is or isn't aligned with human preferences. C. They are only relevant for model pre-training, so they play no role in post-training assessments. D. They serve primarily to reduce the computational cost of fine-tuning a model for alignment.

They provide systematic ways to measure performance and safety, identifying where the model is or isn't aligned with human preferences.

Which statement best describes the role of data curation in post-training alignment for Large Language Models? A. It involves collecting as much text as possible from any source to maximize model exposure. B. It entails selecting, cleaning, and organizing training data to improve quality and reduce biases. C. It uses automated scripts to filter out all content with a negative sentiment score. D. It focuses solely on removing personally identifiable information from the dataset.

It entails selecting, cleaning, and organizing training data to improve quality and reduce biases.

Which of the following statements best captures a common challenge when applying safety fine-tuning to Large Language Models? A. The model often becomes more likely to produce repetitive or overcautious responses, creating a trade-off with helpfulness or creativity. B. Safety fine-tuning requires minimal domain-specific data and does not involve specialized annotation. C. Safety fine-tuning is typically integrated seamlessly with RLHF, requiring no separate datasets. D. Safety fine-tuning primarily reduces the computational cost of model inference.

The model often becomes more likely to produce repetitive or overcautious responses, creating a trade-off with helpfulness or creativity.

In the context of Large Language Model (LLM) training, which statement best captures the idea of power laws as described in scaling laws? Options: They predict that model performance increases linearly with the number of parameters or training data., They show that model performance has a strict upper limit once the dataset exceeds a certain size., They describe how changes in performance scale as a power (exponent) of changes in model size or dataset size., They apply only to small models, becoming irrelevant for models with billions of parameters.

They describe how changes in performance scale as a power (exponent) of changes in model size or dataset size.

AI Model Pre-training Flashcard Flashcards

Student preview

25 questions

Show all answers

1.

FLASHCARD QUESTION

Front

What is the main aim of pre-training in AI models?

Back

To build a general model that can be adapted to various tasks

2.

FLASHCARD QUESTION

Front

Which is the most common method used in pre-training GPAI models? Options: Decision trees, Monte Carlo methods, Reinforcement Learning with Human Feedback (RLHF), Self-supervised learning

Back

Self-supervised learning

3.

FLASHCARD QUESTION

Front

Which three fundamental components are typically referred to as the AI triad, crucial for determining an AI model's capabilities?

Back

Data, compute, and algorithms

4.

FLASHCARD QUESTION

Front

What data type is typically used in pre-training large language models?

Back

Both structured and unstructured internet-scale data

5.

FLASHCARD QUESTION

Front

What is the difference between pre-training and fine-tuning?

Back

Pre-training creates general representations, while fine-tuning adapts the model to particular tasks.

6.

FLASHCARD QUESTION

Front

Which is a key challenge in pre-training AI models?

Back

Resource requirements and computational cost

7.

FLASHCARD QUESTION

Front

Which of these is least likely to be treated as a single token by a large language model? The, 5, Mississippi, Alex

Back

Mississippi

Access all questions and much more by creating a free account

Create resources

Host any resource

Get auto-graded reports

Continue with Google

Continue with Email

Continue with Classlink

Continue with Clever

or continue with

Microsoft

Apple

Others

Already have an account?

Similar Resources on Wayground

20 questions

Grade 9 Modals

Flashcard

•

9th Grade

19 questions

Review for Habitat and food chain

Flashcard

•

2nd Grade

19 questions

Past participle verbs 1

Flashcard

•

University

19 questions

Unit 5: Being Ill and the Experience of Stress and Pain

Flashcard

•

University

19 questions

Articles the/a/an/-

Flashcard

•

8th - 9th Grade

18 questions

Density and Buoyancy

Flashcard

•

6th Grade

20 questions

Database Technologies, Relational Model, and Data Modeling

Flashcard

•

University

20 questions

Magnetic Field

Flashcard

•

12th Grade

Popular Resources on Wayground

15 questions

Fractions on a Number Line

Quiz

•

3rd Grade

20 questions

Equivalent Fractions

Quiz

•

3rd Grade

25 questions

Multiplication Facts

Quiz

•

5th Grade

54 questions

Analyzing Line Graphs & Tables

Quiz

•

4th Grade

$fractions$

22 questions

fractions

Quiz

•

3rd Grade

20 questions

Main Idea and Details

Quiz

•

5th Grade

20 questions

Context Clues

Quiz

•

6th Grade

15 questions

Equivalent Fractions

Quiz

•

4th Grade

Discover more resources for Professional Development

10 questions

SMART Goals

Quiz

•

8th - 12th Grade