Ten questions at a time, drawn from 180. Every answer is explained. Nothing is saved and no account is needed.
When building an LLM from scratch, why is unsupervised learning typically preferred over supervised learning for pretraining?
Practice quiz for . Scores are not saved.
When building an LLM from scratch, why is unsupervised learning typically preferred over supervised learning for pretraining?
Answer: Unsupervised learning leverages vast amounts of unlabeled text, making it more scalable and cost-effective for pretraining.. The correct answer is that unsupervised learning leverages vast amounts of unlabeled text, making it more scalable and cost-effective for pretraining. Labeled data is expensive and impractical for pretraining LLMs, while unsupervised objectives like masked language modeling or next-word prediction work well with raw text. The other options are incorrect: (0) Unsupervised learning does not require labeled data; (1) Supervised learning is not computationally cheaper for pretraining; (3) Supervised learning is not inherently better for language understanding—it depends on the task.
From lesson: Introduction to Machine Learning: Supervised vs. Unsupervised Learning
You are fine-tuning an LLM for a chatbot application. Which approach is most appropriate for this stage?
Answer: Use supervised learning to train the model on labeled conversational data.. The correct answer is to use supervised learning to train the model on labeled conversational data. Fine-tuning aligns the pretrained LLM with the specific task (e.g., chatbot responses) using labeled examples. The other options are incorrect: (0) Unsupervised learning is for pretraining, not fine-tuning; (2) Skipping fine-tuning results in generic outputs; (3) RLHF is often used after supervised fine-tuning, not as a replacement.
From lesson: Introduction to Machine Learning: Supervised vs. Unsupervised Learning
What is a key advantage of using self-supervised learning (a type of unsupervised learning) for pretraining an LLM?
Answer: It allows the model to learn from raw text without requiring labeled data.. The correct answer is that self-supervised learning allows the model to learn from raw text without requiring labeled data. This makes it scalable and practical for pretraining LLMs. The other options are incorrect: (0) Self-supervised learning still requires human-designed objectives (e.g., masked language modeling); (2) It does not guarantee better performance on all tasks; (3) It does not inherently reduce model size or inference costs.
From lesson: Introduction to Machine Learning: Supervised vs. Unsupervised Learning
You observe that your pretrained LLM performs poorly on a summarization task. What is the most likely reason and fix?
Answer: The model was not fine-tuned on labeled summarization data. Fix: Fine-tune the model on a summarization dataset.. The correct answer is that the model was not fine-tuned on labeled summarization data. Pretraining provides general language understanding, but fine-tuning is necessary to adapt the model to specific tasks like summarization. The other options are incorrect: (0) Pretraining is typically unsupervised, not supervised; (2) Increasing model size may help but is not the most likely fix; (3) Pretraining data does not need to include task-specific examples—fine-tuning handles that.
From lesson: Introduction to Machine Learning: Supervised vs. Unsupervised Learning
Why might a developer choose to use supervised fine-tuning after unsupervised pretraining for an LLM?
Answer: Supervised fine-tuning aligns the model with specific tasks or domains using labeled data.. The correct answer is that supervised fine-tuning aligns the model with specific tasks or domains using labeled data. Pretraining provides broad language understanding, while fine-tuning adapts the model to particular applications (e.g., chatbots, translation). The other options are incorrect: (0) Supervised fine-tuning is not necessarily faster; (1) Unsupervised pretraining can learn useful patterns from raw text; (3) Unsupervised pretraining is widely used for language tasks.
From lesson: Introduction to Machine Learning: Supervised vs. Unsupervised Learning
Why are deep neural networks preferred over shallow networks for language modeling tasks?
Answer: Deep networks can learn hierarchical features, capturing complex patterns like syntax and semantics more effectively.. The correct answer is that deep networks can learn hierarchical features, which is essential for capturing the layered structure of language (e.g., characters → words → phrases → meaning). The other options are incorrect because: 1) Deep networks require more computational power, not less. 3) Shallow networks struggle with complex patterns in language. 4) Deep networks often rely on attention mechanisms to improve performance.
From lesson: Basics of Neural Networks and Deep Learning
What is the primary purpose of using an attention mechanism in a language model?
Answer: To allow the model to focus on different parts of the input sequence dynamically, depending on the context.. The correct answer is that attention mechanisms allow the model to focus on different parts of the input sequence dynamically, which is crucial for understanding context-dependent relationships in language. The other options are incorrect because: 1) Attention mechanisms increase parameters, not reduce them. 3) Attention does not replace embeddings; they serve different purposes. 4) Attention explicitly weighs tokens differently, not equally.
From lesson: Basics of Neural Networks and Deep Learning
During backpropagation in a deep neural network for language modeling, you notice the gradients in early layers are extremely small. What is the most likely cause and solution?
Answer: The activation functions are causing vanishing gradients. Replace sigmoid/tanh with ReLU or use residual connections.. The correct answer is that the activation functions (e.g., sigmoid/tanh) are causing vanishing gradients, and replacing them with ReLU or using residual connections can mitigate this. The other options are incorrect because: 1) Small gradients are a sign of vanishing, not exploding, gradients. 3) Overfitting is unrelated to gradient size. 4) Batch size affects gradient stability but is not the primary cause of vanishing gradients.
From lesson: Basics of Neural Networks and Deep Learning
You are training a language model and observe that it performs well on the training data but poorly on unseen text. What is the most effective way to address this?
Answer: Apply regularization techniques like dropout or weight decay to improve generalization.. The correct answer is to apply regularization techniques like dropout or weight decay, which prevent the model from overfitting to the training data and improve generalization. The other options are incorrect because: 1) Increasing model size without regularization can worsen overfitting. 3) Reducing the learning rate may not address overfitting. 4) More epochs can exacerbate overfitting by memorizing training data.
From lesson: Basics of Neural Networks and Deep Learning
Why is the transformer architecture particularly well-suited for language modeling compared to traditional RNNs?
Answer: Transformers process the entire input sequence in parallel, enabling faster training and better capture of long-range dependencies.. The correct answer is that transformers process the entire input sequence in parallel, which speeds up training and better captures long-range dependencies in language. The other options are incorrect because: 1) Transformers do not use recurrent connections; they rely on self-attention. 3) Transformers typically have more parameters, not fewer. 4) Transformers still use embeddings for input representation.
From lesson: Basics of Neural Networks and Deep Learning
When building a tokenizer for your own LLM, why is subword tokenization (e.g., BPE) preferred over character-level tokenization?
Answer: Subword tokenization balances vocabulary size and rare word handling, reducing out-of-vocabulary issues while maintaining efficiency.. The correct answer is that subword tokenization balances vocabulary size and rare word handling. Character-level tokenization (option 0) is inefficient for large vocabularies and loses semantic meaning. Option 2 is incorrect because subword tokenization is language-agnostic. Option 3 is wrong because character-level tokenization does not preserve word boundaries or semantic meaning effectively.
From lesson: Natural Language Processing (NLP) Fundamentals
During pretraining of your LLM, you notice the model struggles with long-range dependencies. Which architectural choice is most likely to address this issue?
Answer: Using positional embeddings and multi-head self-attention to model relationships across distant tokens.. The correct answer is using positional embeddings and multi-head self-attention. Positional embeddings encode token order, and self-attention captures long-range dependencies. Option 0 (increasing embedding dimension) doesn’t address dependencies. Option 1 (RNNs) is worse for long-range dependencies. Option 3 (reducing sequence length) avoids the problem rather than solving it.
From lesson: Natural Language Processing (NLP) Fundamentals
You’re fine-tuning your LLM for a text classification task. Which loss function should you use, and why?
Answer: Cross-entropy loss, because it is designed for classification tasks and penalizes incorrect predictions more effectively.. The correct answer is cross-entropy loss. It is the standard for classification tasks, providing better gradients for discrete outputs. MSE (option 0) is for regression. Hinge loss (option 2) is less common for multi-class tasks. KL divergence (option 3) is used for generative modeling, not classification.
From lesson: Natural Language Processing (NLP) Fundamentals
Your LLM’s embeddings are not capturing contextual nuances (e.g., ‘bank’ as a financial institution vs. riverbank). What is the most effective way to address this?
Answer: Leverage the transformer’s self-attention mechanism to generate contextual embeddings dynamically.. The correct answer is leveraging self-attention to generate contextual embeddings. Static embeddings (option 0) cannot disambiguate context. A larger vocabulary (option 1) doesn’t solve polysemy. One-hot encodings (option 3) lose all semantic information.
From lesson: Natural Language Processing (NLP) Fundamentals
You’re designing a dataset for pretraining your LLM. Why is it important to filter out low-quality or biased data?
Answer: Biased data can lead to unfair or harmful outputs, and low-quality data introduces noise that degrades model performance.. The correct answer is that biased or low-quality data harms performance and fairness. Option 0 is wrong because quality directly impacts performance. Option 2 is incorrect because models amplify biases. Option 3 is false because pretraining quality is critical for downstream tasks.
From lesson: Natural Language Processing (NLP) Fundamentals
Why is subword tokenization (e.g., BPE) preferred over word-level tokenization for building an LLM?
Answer: It handles rare and unseen words by breaking them into known subword units, preserving semantic meaning.. The correct answer is that subword tokenization handles rare and unseen words by breaking them into known subword units. This preserves semantic meaning and avoids out-of-vocabulary issues. The other options are incorrect: subword tokenization does not reduce vocabulary size to words (it often increases it for subwords), it does not eliminate normalization, and it does not guarantee perfect tokenization for all languages.
From lesson: Tokenization and Text Preprocessing Techniques
A student trains an LLM on a dataset where all text is lowercase but forgets to normalize new input text during inference. What is the most likely consequence?
Answer: The model will create separate tokens for uppercase and lowercase versions, potentially reducing performance.. The correct answer is that the model will create separate tokens for uppercase and lowercase versions, potentially reducing performance. Since the training data was lowercase, the tokenizer may not recognize uppercase variants, leading to out-of-vocabulary tokens or redundant entries. The other options are incorrect: the model won't fail entirely, it won't treat them as identical, and it won't normalize input automatically unless explicitly programmed to do so.
From lesson: Tokenization and Text Preprocessing Techniques
During preprocessing, a student removes all punctuation from the text. How might this affect the LLM's performance?
Answer: It may harm performance by losing semantic or syntactic cues (e.g., question marks, commas).. The correct answer is that removing punctuation may harm performance by losing semantic or syntactic cues. Punctuation often carries meaning (e.g., questions vs statements) or affects sentence structure. The other options are incorrect: removing punctuation doesn't always improve performance, it is not irrelevant, and tokenizers can function without it.
From lesson: Tokenization and Text Preprocessing Techniques
A student uses a tokenizer with a vocabulary size of 50,000 for an LLM trained on a small dataset of 10,000 words. What is the most likely issue?
Answer: The vocabulary size is too large, leading to inefficient memory usage and slower training.. The correct answer is that the vocabulary size is too large, leading to inefficient memory usage and slower training. A large vocabulary increases embedding matrix size and computational cost without adding value for a small dataset. The other options are incorrect: 50,000 is not too small for 10,000 words, it doesn't match the dataset size, and tokenizers don't dynamically adjust vocabulary size.
From lesson: Tokenization and Text Preprocessing Techniques
Why is it important to include special tokens like [PAD] and [UNK] in an LLM's tokenizer?
Answer: They handle padding for variable-length sequences and unknown tokens during training/inference.. The correct answer is that special tokens handle padding for variable-length sequences ([PAD]) and unknown tokens ([UNK]). This ensures the model can process batches of text and handle out-of-vocabulary words. The other options are incorrect: special tokens aren't required for generation, they don't replace punctuation/whitespace, and they don't guarantee multilingual support.
From lesson: Tokenization and Text Preprocessing Techniques
When building an LLM for a domain with many rare words (e.g., medical texts), which Word2Vec architecture is most suitable and why?
Answer: Skip-gram, because it predicts context from a target word and handles rare words better. The correct answer is Skip-gram. Skip-gram trains on individual word-context pairs, giving rare words more exposure during training, which is critical for domains with specialized vocabulary. CBOW averages context vectors, which dilutes the signal for rare words. The other options are incorrect because Word2Vec *can* handle rare words (especially Skip-gram), and the architectures are not identical in performance.
From lesson: Word Embeddings: Word2Vec, GloVe, and FastText
Your LLM needs to generate text in a morphologically rich language (e.g., Finnish). Why might FastText be a better choice than GloVe for the embedding layer?
Answer: FastText includes subword (n-gram) embeddings, which capture morphological variations and improve handling of rare words. The correct answer is that FastText includes subword embeddings. This allows it to represent rare or morphologically complex words (e.g., conjugations, compounds) by breaking them into n-grams, which is critical for morphologically rich languages. The other options are wrong: GloVe works for any language, training speed depends on implementation, and GloVe supports arbitrary dimensions.
From lesson: Word Embeddings: Word2Vec, GloVe, and FastText
You observe that your LLM's embeddings perform well on word similarity tasks but poorly on a downstream classification task. What is the most likely explanation?
Answer: The embeddings were not fine-tuned on the classification task's domain-specific corpus. The correct answer is that the embeddings were not fine-tuned on the domain-specific corpus. Intrinsic tasks (e.g., word similarity) often don't reflect performance on downstream tasks, especially if the embeddings lack domain-specific knowledge. While the other options may contribute, fine-tuning is the most direct fix for this mismatch. Window size and vocabulary are secondary concerns.
From lesson: Word Embeddings: Word2Vec, GloVe, and FastText
During LLM training, you notice that words with similar meanings (e.g., 'happy' and 'joyful') are close in the embedding space, but words with opposite meanings (e.g., 'happy' and 'sad') are also close. What is the most likely cause?
Answer: The embedding layer was trained with a very large context window, causing antonyms to appear in similar contexts. The correct answer is a very large context window. Large windows capture broad semantic relationships, which can cause antonyms (e.g., 'happy' and 'sad') to appear in similar contexts (e.g., 'I feel ___ today'). Small windows focus on syntax, and the other options don't directly explain the clustering of opposites. The loss function and vocabulary size would affect overall performance, not specifically antonyms.
From lesson: Word Embeddings: Word2Vec, GloVe, and FastText
You are designing an LLM for a social media platform where text contains many misspellings (e.g., 'goood' instead of 'good'). Which embedding method is most robust to such noise?
Answer: FastText, because it uses subword embeddings to represent misspelled words as combinations of n-grams. The correct answer is FastText. Its subword embeddings allow it to represent misspelled words (e.g., 'goood') as combinations of n-grams (e.g., 'goo', 'ood'), making it robust to noise. Word2Vec and GloVe treat words as atomic units and fail on misspellings. While preprocessing helps, FastText is specifically designed to handle such cases without it.
From lesson: Word Embeddings: Word2Vec, GloVe, and FastText
Why are positional encodings necessary in transformer architectures?
Answer: They enable the model to process sequential data by providing information about token positions, which transformers lack inherently.. The correct answer is that positional encodings provide information about token positions, which transformers inherently lack due to their non-recurrent design. The other options are incorrect: positional encodings do not replace attention, reduce complexity, or normalize embeddings.
From lesson: Introduction to Transformers and Attention Mechanisms
What is the primary advantage of multi-head attention over single-head attention?
Answer: It allows the model to focus on different parts of the input simultaneously, capturing diverse relationships.. The correct answer is that multi-head attention allows the model to focus on different parts of the input simultaneously, capturing diverse relationships. The other options are incorrect: multi-head attention increases parameters, does not eliminate positional encodings, and does not enforce sparsity.
From lesson: Introduction to Transformers and Attention Mechanisms
Why are residual connections important in transformer architectures?
Answer: They enable deeper architectures by mitigating vanishing gradients and allowing gradients to flow directly through the network.. The correct answer is that residual connections enable deeper architectures by mitigating vanishing gradients and allowing gradients to flow directly. The other options are incorrect: residual connections do not replace layer normalization, reduce memory, or enforce sparsity.
From lesson: Introduction to Transformers and Attention Mechanisms
What is a key limitation of standard attention mechanisms in transformers?
Answer: They require quadratic memory and compute relative to sequence length, limiting scalability for long sequences.. The correct answer is that standard attention mechanisms require quadratic memory and compute relative to sequence length, limiting scalability. The other options are incorrect: transformers process sequential data without recurrence, do not rely on recurrence, and can capture long-range dependencies.
From lesson: Introduction to Transformers and Attention Mechanisms
How does the softmax function contribute to the attention mechanism?
Answer: It normalizes attention scores into probabilities, ensuring they sum to 1 and represent valid importance weights.. The correct answer is that softmax normalizes attention scores into probabilities, ensuring they sum to 1. The other options are incorrect: softmax does not reduce dimensionality, replace positional encodings, or enforce sparsity.
From lesson: Introduction to Transformers and Attention Mechanisms
Why do n-gram language models struggle with capturing long-range dependencies in text?
Answer: They rely on fixed-size windows of previous words, limiting context to a few tokens.. The correct answer is that n-gram models rely on fixed-size windows, which restricts their ability to model context beyond a few tokens. The other options are incorrect because: (1) Neural networks, not n-grams, learn dependencies across entire sequences. (2) Smoothing helps with unseen n-grams but doesn't address long-range dependencies. (3) N-gram models generalize from training data, not memorize documents.
From lesson: Overview of Language Models: From N-grams to Neural LMs
What is the primary purpose of smoothing techniques in n-gram language models?
Answer: To assign non-zero probabilities to unseen n-grams and improve generalization.. The correct answer is that smoothing assigns non-zero probabilities to unseen n-grams, improving generalization. The other options are incorrect because: (1) Smoothing doesn't increase vocabulary size. (2) It doesn't reduce computational cost. (3) It doesn't address long-range dependencies, which are a limitation of n-gram models.
From lesson: Overview of Language Models: From N-grams to Neural LMs
How does subword tokenization (e.g., BPE) benefit neural language models compared to word-level tokenization?
Answer: It handles rare words and morphological variations by breaking words into subword units.. The correct answer is that subword tokenization handles rare words and morphological variations by breaking words into subword units. The other options are incorrect because: (1) Subword tokenization typically increases vocabulary size. (2) Embeddings are still required for subword units. (3) Neural models generalize, not memorize, word forms.
From lesson: Overview of Language Models: From N-grams to Neural LMs
Why might a developer choose a smaller neural language model over a larger one for a custom LLM project?
Answer: Smaller models require fewer resources (data, compute, time) and may suffice for the task.. The correct answer is that smaller models require fewer resources and may suffice for the task. The other options are incorrect because: (1) Larger models often generalize better with sufficient data. (2) Both small and large models can capture long-range dependencies, but larger ones do it better. (3) Tokenization and embeddings are still needed regardless of model size.
From lesson: Overview of Language Models: From N-grams to Neural LMs
What is a key advantage of neural language models (e.g., RNNs, Transformers) over traditional n-gram models?
Answer: They can learn dependencies across entire sequences through embeddings and attention mechanisms.. The correct answer is that neural language models learn dependencies across entire sequences using embeddings and attention. The other options are incorrect because: (1) Fixed-size windows are a limitation of n-gram models, not an advantage of neural models. (2) Neural models still require tokenization. (3) Neural models assign non-zero probabilities to unseen sequences through generalization.
From lesson: Overview of Language Models: From N-grams to Neural LMs
When implementing a Transformer decoder for your own LLM, why is causal masking essential during training?
Answer: It prevents the model from attending to future tokens, ensuring autoregressive generation mimics inference conditions.. The correct answer is that causal masking prevents the model from attending to future tokens, ensuring the training process mirrors inference (where future tokens are unknown). The other options are incorrect: (1) While causal masking does limit attention, its primary purpose is not computational efficiency. (2) Causal masking enforces unidirectional context, unlike BERT's bidirectional approach. (3) Causal masking does not replace positional encodings; they serve orthogonal purposes (order vs. context).
From lesson: Key Papers in NLP: Attention Is All You Need, BERT, and GPT
You are fine-tuning BERT for a custom LLM task. Why might freezing the lower layers and only training the top layers lead to poor performance?
Answer: Lower layers capture general language features (e.g., syntax), while higher layers specialize in task-specific patterns; freezing them prevents adaptation.. The correct answer is that lower layers capture general language features, while higher layers specialize in task-specific patterns. Freezing lower layers prevents the model from adapting these foundational features to the new task. The other options are incorrect: (1) Frozen layers do propagate gradients (they just don't update their weights). (2) BERT can be fine-tuned with partial layer training. (3) Lower layers are not task-specific; they learn general representations.
From lesson: Key Papers in NLP: Attention Is All You Need, BERT, and GPT
In 'Attention Is All You Need,' why is the scaled dot-product attention mechanism preferred over additive attention for Transformers?
Answer: Scaled dot-product attention is more computationally efficient and parallelizable on GPUs, despite similar theoretical expressiveness.. The correct answer is that scaled dot-product attention is more computationally efficient and parallelizable, which is critical for training large models. The other options are incorrect: (1) Additive attention can handle variable-length sequences. (2) Scaled dot-product attention does not include positional encodings. (3) Both attention mechanisms use quadratic memory for the attention matrix.
From lesson: Key Papers in NLP: Attention Is All You Need, BERT, and GPT
When designing your own LLM, you notice that GPT's autoregressive generation produces repetitive text. Which modification would most directly address this issue?
Answer: Introduce a penalty for repeating n-grams during decoding, such as nucleus sampling with repetition constraints.. The correct answer is to introduce a penalty for repeating n-grams, such as nucleus sampling with repetition constraints. This directly targets the issue of repetition. The other options are incorrect: (1) Bidirectional attention would break autoregressive generation. (2) Increasing depth may worsen repetition by overfitting. (3) Removing positional encodings would harm coherence, not fix repetition.
From lesson: Key Papers in NLP: Attention Is All You Need, BERT, and GPT
Why does BERT's masked language modeling (MLM) objective make it unsuitable for direct autoregressive text generation?
Answer: MLM trains the model to predict masked tokens using bidirectional context, while autoregressive generation requires unidirectional context.. The correct answer is that MLM trains the model to predict masked tokens using bidirectional context, while autoregressive generation requires unidirectional context (predicting the next token based only on past tokens). The other options are incorrect: (1) BERT can generate text with modifications (e.g., BART), but its training objective is not autoregressive. (2) MLM does not inherently conflict with attention mechanisms. (3) BERT can be adapted for generation tasks, though it is not its primary design.
From lesson: Key Papers in NLP: Attention Is All You Need, BERT, and GPT
When building a custom LLM for a translation task, why is an encoder-decoder architecture typically preferred over a decoder-only model?
Answer: Encoder-decoder models explicitly separate input processing (encoder) from output generation (decoder), better handling variable-length input-output mappings.. The correct answer is that encoder-decoder models explicitly separate input processing from output generation, which is ideal for tasks like translation where input and output lengths differ. The encoder compresses the input into a context vector, while the decoder generates the output autoregressively. Option 1 is wrong because encoder-decoder models are often more complex. Option 2 is incorrect because decoder-only models can generate sequences of arbitrary length. Option 4 is false because decoder-only models do use attention (e.g., causal self-attention).
From lesson: Architecture of Transformer Models: Encoder-Decoder vs. Decoder-Only
In a decoder-only LLM, what is the primary purpose of the causal attention mask?
Answer: To ensure the model only attends to tokens before the current position, preventing future token leakage.. The correct answer is that the causal attention mask ensures the model only attends to prior tokens, preventing it from 'cheating' by looking ahead during training. This enforces autoregressive behavior. Option 1 is incorrect because masks don’t reduce computational overhead. Option 3 is wrong because masks enforce unidirectional context, not bidirectional. Option 4 is false because masks and positional encodings serve different purposes.
From lesson: Architecture of Transformer Models: Encoder-Decoder vs. Decoder-Only
You are designing a custom LLM for a chatbot that must generate responses based on user input. Why might a decoder-only architecture be a better choice than an encoder-decoder model?
Answer: Encoder-decoder models require paired input-output data, while decoder-only models can be fine-tuned on unstructured text.. The correct answer is that decoder-only models can be fine-tuned on unstructured text (e.g., next-token prediction), while encoder-decoder models often require paired input-output data (e.g., translation pairs). This makes decoder-only models more flexible for chatbots. Option 1 is wrong because decoder-only models are unidirectional. Option 3 is incorrect because both architectures can generate output in a single pass. Option 4 is false because encoder-decoder models can generate long sequences.
From lesson: Architecture of Transformer Models: Encoder-Decoder vs. Decoder-Only
What is a key limitation of decoder-only models when compared to encoder-decoder models for tasks like summarization?
Answer: Decoder-only models lack the ability to process the entire input document bidirectionally before generating output.. The correct answer is that decoder-only models lack bidirectional context, which is critical for summarization tasks where the model must understand the entire input before generating a concise output. Encoder-decoder models process the input bidirectionally in the encoder. Option 1 is incorrect because decoder-only models can generate outputs of arbitrary length. Option 3 is false because inference speed depends on implementation, not architecture. Option 4 is wrong because cross-attention is a feature of encoder-decoder models, not a limitation of decoder-only models.
From lesson: Architecture of Transformer Models: Encoder-Decoder vs. Decoder-Only
When implementing a custom LLM, you notice that your decoder-only model’s outputs are repetitive and lack coherence. Which of the following is the most likely cause?
Answer: The positional encodings are missing or improperly applied, causing the model to lose track of token order.. The correct answer is that missing or improper positional encodings can cause the model to lose track of token order, leading to repetitive or incoherent outputs. Positional encodings are critical for decoder-only models to understand sequence structure. Option 1 is irrelevant because the model is confirmed to be decoder-only. Option 2 is less likely because incorrect masks would cause training instability, not necessarily repetitive outputs. Option 4 could cause issues but is not the most likely cause of repetition.
From lesson: Architecture of Transformer Models: Encoder-Decoder vs. Decoder-Only
In self-attention, why do we scale the dot product of queries and keys by the square root of the key dimension?
Answer: To prevent the softmax from saturating due to large dot product values, which would cause gradient vanishing. The correct answer is the first option: scaling prevents softmax saturation by keeping dot products in a reasonable range, which maintains gradient flow during backpropagation. The second option is incorrect because softmax already normalizes weights. The third option is wrong because scaling adds minimal computation. The fourth option is incorrect because softmax inherently produces positive probabilities, regardless of scaling.
From lesson: Self-Attention Mechanism and Multi-Head Attention
What is the primary advantage of using multi-head attention over single-head attention in an LLM?
Answer: It allows the model to jointly attend to information from different representation subspaces at different positions. The correct answer is the first option: multi-head attention enables the model to capture diverse patterns by learning distinct query, key, and value projections for each head. The second option is incorrect because multi-head attention increases parameters. The third option is partially true but not the primary advantage. The fourth option is wrong because attention heads are not inherently interpretable.
From lesson: Self-Attention Mechanism and Multi-Head Attention
During training of an autoregressive LLM, why must future tokens be masked in the decoder's self-attention?
Answer: To prevent the model from 'cheating' by attending to future tokens, which would break the autoregressive property. The correct answer is the first option: masking future tokens ensures the model only attends to past and current tokens, preserving the autoregressive nature of generation. The second option is incorrect because masking doesn't reduce memory usage. The third option is wrong because masking isn't about stabilization. The fourth option is incorrect because masking doesn't enforce a fixed window size.
From lesson: Self-Attention Mechanism and Multi-Head Attention
How does the self-attention mechanism dynamically adapt to different input sequences?
Answer: By computing attention weights as a function of the input tokens, allowing different tokens to attend to each other based on their content. The correct answer is the first option: self-attention dynamically computes attention weights via query-key dot products, making it input-dependent. The second option describes static weights, not self-attention. The third option is incorrect because weights are learned, not random. The fourth option is wrong because normalization doesn't enable dynamic adaptation.
From lesson: Self-Attention Mechanism and Multi-Head Attention
In multi-head attention, what is the purpose of concatenating the outputs of all heads and projecting them through a final linear layer?
Answer: To combine the diverse patterns learned by each head into a unified representation for the next layer. The correct answer is the first option: concatenation and projection integrate the specialized representations from each head into a cohesive output. The second option is incorrect because dimensionality reduction isn't the primary goal. The third option is wrong because heads don't contribute equally. The fourth option is incorrect because normalization isn't the purpose of this step.
From lesson: Self-Attention Mechanism and Multi-Head Attention
Why is positional encoding necessary in transformer-based LLMs?
Answer: It provides the model with information about the order of tokens in the sequence, which is otherwise lost in self-attention.. The correct answer is that positional encoding provides order information. Self-attention mechanisms are permutation-invariant, meaning they don't inherently understand token order. Positional encoding injects this missing information. The other options are incorrect: positional encoding doesn't help with memorization, doesn't reduce computational cost, and doesn't replace token embeddings.
From lesson: Positional Encoding and Its Importance
What is a key advantage of using sinusoidal functions for positional encoding in LLMs?
Answer: They enable the model to generalize to sequence lengths not seen during training by leveraging relative position patterns.. The correct answer is that sinusoidal functions enable generalization to unseen sequence lengths. The periodic nature of sine and cosine functions allows the model to infer relative positions even for sequences longer than those in the training data. The other options are incorrect: sinusoidal functions don't enable memorization, don't eliminate learned embeddings, and don't restrict the model to fixed-length sequences.
From lesson: Positional Encoding and Its Importance
How does positional encoding interact with token embeddings in a transformer model?
Answer: Positional encodings are added to token embeddings element-wise, combining order and semantic information.. The correct answer is that positional encodings are added to token embeddings element-wise. This combines the semantic information from token embeddings with the positional information. The other options are incorrect: positional encodings are not concatenated, they don't replace token embeddings, and they are not multiplied with them.
From lesson: Positional Encoding and Its Importance
What happens if you remove positional encoding from a transformer-based LLM?
Answer: The model will fail to distinguish the order of tokens, leading to poor performance on tasks requiring sequential understanding.. The correct answer is that the model will fail to distinguish token order. Self-attention is permutation-invariant, so without positional encoding, the model cannot understand the sequence of tokens. The other options are incorrect: self-attention does not inherently understand order, removing positional encoding doesn't improve efficiency, and the model cannot learn positional information without explicit encoding.
From lesson: Positional Encoding and Its Importance
Why might a learned positional embedding be preferred over sinusoidal positional encoding in some LLMs?
Answer: Learned embeddings can adapt to the specific patterns in the training data, potentially improving performance.. The correct answer is that learned embeddings can adapt to the training data's specific patterns. Unlike sinusoidal functions, learned embeddings are optimized during training and can capture dataset-specific positional patterns. The other options are incorrect: learned embeddings add parameters, don't guarantee generalization to longer sequences, and don't replace token embeddings.
From lesson: Positional Encoding and Its Importance
Why is masked language modeling (MLM) considered more challenging than standard language modeling for training LLMs?
Answer: MLM forces the model to rely on bidirectional context, making it harder to exploit local patterns.. The correct answer is that MLM forces the model to rely on bidirectional context. Unlike standard language modeling (which predicts the next token using left context), MLM masks random tokens and requires the model to infer them using both left and right context, making it harder to 'cheat' with local patterns. The other options are incorrect: MLM predicts one masked token at a time (not multiple), it’s a pre-training objective, and it doesn’t restrict vocabulary.
From lesson: Training Objectives: Language Modeling, Masked Language Modeling, and Next Sentence Prediction
A student trains an LLM using next sentence prediction (NSP) but finds the model performs poorly on downstream tasks. What is the most likely cause?
Answer: The negative samples were too easy (e.g., sentences from unrelated documents).. The correct answer is that the negative samples were too easy. If negatives are trivial (e.g., sentences from unrelated documents), the model doesn’t learn meaningful relationships between sentences. Hard negatives (e.g., semantically plausible but non-consecutive sentences) are critical for NSP’s effectiveness. The other options are incorrect: Too many positive pairs would bias the model but not necessarily hurt performance; NSP doesn’t require a separate head; and learning rate is a secondary concern.
From lesson: Training Objectives: Language Modeling, Masked Language Modeling, and Next Sentence Prediction
How does the choice of masking strategy in MLM impact the model’s ability to handle rare words?
Answer: Dynamic masking increases the likelihood of masking rare words, improving their representations.. The correct answer is that dynamic masking increases the likelihood of masking rare words. Dynamic masking (e.g., changing masked tokens in each epoch) ensures the model sees rare words in masked positions more often, forcing it to learn better representations. The other options are incorrect: Whole-word masking doesn’t exclude rare words; span masking can include rare words; and random masking does affect rare words (they’re less likely to be masked due to lower frequency).
From lesson: Training Objectives: Language Modeling, Masked Language Modeling, and Next Sentence Prediction
A student observes that their LLM trained with MLM achieves low perplexity but generates incoherent text. What is the most probable explanation?
Answer: The masking strategy didn’t account for sentence boundaries, disrupting coherence.. The correct answer is that the masking strategy didn’t account for sentence boundaries. If masks frequently split phrases or clauses, the model struggles to learn coherent structures. The other options are incorrect: High masking rates don’t inherently cause incoherence; attention is critical for MLM; and combining MLM with LM doesn’t create conflicts (they’re often used together).
From lesson: Training Objectives: Language Modeling, Masked Language Modeling, and Next Sentence Prediction
Why might a student choose to omit next sentence prediction (NSP) when training their own LLM, despite its original inclusion in models like BERT?
Answer: Recent research shows NSP doesn’t improve performance on downstream tasks like question answering.. The correct answer is that recent research shows NSP doesn’t improve performance on downstream tasks. Studies (e.g., RoBERTa) found that removing NSP and focusing on MLM with larger batches and longer sequences yields better results. The other options are incorrect: NSP doesn’t require labeled data (it uses heuristics); it’s not computationally expensive; and it’s compatible with relative positional embeddings.
From lesson: Training Objectives: Language Modeling, Masked Language Modeling, and Next Sentence Prediction
When building a custom LLM for a specialized domain (e.g., legal contracts), why is fine-tuning a pre-trained model preferred over training from scratch?
Answer: Fine-tuning requires less computational power and data because it leverages pre-existing language knowledge.. The correct answer is that fine-tuning requires less computational power and data because it builds on pre-trained knowledge. The other options are incorrect: training from scratch is possible with sufficient resources (though impractical), fine-tuning doesn't guarantee higher accuracy for all cases, and pre-trained models still need adaptation for specialized domains.
From lesson: Fine-Tuning vs. Pre-Training: Transfer Learning in NLP
A student fine-tunes an LLM on a small dataset (1,000 examples) but notices the model performs poorly on unseen data. What is the most likely cause?
Answer: The model overfit the small dataset, memorizing noise instead of generalizing.. The correct answer is overfitting, where the model memorizes the small dataset instead of learning generalizable patterns. The other options are less likely: low learning rates would underfit, not overfit; model size isn't the primary issue here; and dataset diversity is usually beneficial.
From lesson: Fine-Tuning vs. Pre-Training: Transfer Learning in NLP
During fine-tuning, a developer freezes the first 8 layers of a 12-layer LLM and trains only the top 4 layers. What is the primary benefit of this approach?
Answer: It prevents catastrophic forgetting by preserving general language features in lower layers.. The correct answer is preventing catastrophic forgetting by preserving general features. Freezing lower layers retains pre-trained knowledge while allowing higher layers to adapt. The other options are incorrect: reduced training time is a secondary benefit, freezing doesn't guarantee no overfitting, and the model doesn't learn entirely new rules—it adapts existing ones.
From lesson: Fine-Tuning vs. Pre-Training: Transfer Learning in NLP
A team fine-tunes an LLM for sentiment analysis but finds the model performs worse than the original pre-trained version. What is the most probable explanation?
Answer: The learning rate was too high, overwriting useful pre-trained weights.. The correct answer is a high learning rate, which can destroy pre-trained knowledge. The other options are unlikely: large datasets typically help fine-tuning, pre-trained models aren't perfect for specific tasks, and architecture incompatibility isn't the issue here.
From lesson: Fine-Tuning vs. Pre-Training: Transfer Learning in NLP
Why might a developer choose to use gradual unfreezing (unfreezing layers one by one) instead of fine-tuning all layers at once?
Answer: It reduces the risk of overfitting by stabilizing training and preserving pre-trained knowledge.. The correct answer is reducing overfitting risk by stabilizing training. Gradual unfreezing helps retain pre-trained knowledge while adapting to the task. The other options are incorrect: it doesn't speed up learning, still requires validation, and doesn't guarantee global optima.
From lesson: Fine-Tuning vs. Pre-Training: Transfer Learning in NLP
When scaling a language model, which of the following is the MOST critical factor to balance with model size for optimal performance?
Answer: The quality and diversity of the training data. The correct answer is the quality and diversity of the training data. Scaling laws emphasize that model performance depends on the interplay of model size, data, and compute. Poor-quality or insufficient data will limit performance regardless of model size. The other options are irrelevant: GPU count is a compute resource (not the most critical factor alone), the programming language doesn’t affect scaling, and the dashboard’s color scheme is trivial.
From lesson: Scaling Laws: Model Size, Data, and Compute
A team trains a 10B parameter model on a dataset of 100B tokens and achieves good performance. They then double the model size to 20B parameters but keep the dataset the same. What is the MOST likely outcome?
Answer: Performance will degrade due to overfitting. The correct answer is performance will degrade due to overfitting. Scaling laws show that model size must be balanced with data quantity. Doubling the model size without increasing data leads to overfitting, as the model memorizes the limited data instead of generalizing. The other options are incorrect: performance doesn’t scale linearly with model size, unchanged data doesn’t guarantee unchanged performance, and memory issues aren’t the primary concern here.
From lesson: Scaling Laws: Model Size, Data, and Compute
According to scaling laws, what is the primary purpose of increasing the compute budget for training a language model?
Answer: To support larger model sizes and/or more training data. The correct answer is to support larger model sizes and/or more training data. Scaling laws show that compute enables training larger models or using more data, both of which improve performance. The other options are incorrect: while larger batch sizes can speed up training, they aren’t the primary purpose of compute scaling; compute doesn’t reduce the need for data quality; and validation loss monitoring remains essential regardless of compute.
From lesson: Scaling Laws: Model Size, Data, and Compute
A team observes that their model’s validation loss plateaus despite increasing model size. What is the MOST likely cause?
Answer: The training data is insufficient or of low quality for the model size. The correct answer is the training data is insufficient or of low quality for the model size. Scaling laws indicate that validation loss plateaus when the model’s capacity exceeds the data’s ability to generalize. The other options are incorrect: models don’t have a fixed intelligence limit, a high learning rate would cause instability (not plateauing), and sentience is not a factor in scaling laws.
From lesson: Scaling Laws: Model Size, Data, and Compute
Why is it important to monitor validation loss when scaling a language model?
Answer: To detect overfitting or underfitting and adjust scaling accordingly. The correct answer is to detect overfitting or underfitting and adjust scaling accordingly. Validation loss helps assess generalization performance. If it plateaus or increases, it signals issues like overfitting (model too large for data) or underfitting (model too small). The other options are incorrect: memorization is undesirable, political correctness isn’t a scaling concern, and early stopping is a secondary use of validation loss, not its primary purpose.
From lesson: Scaling Laws: Model Size, Data, and Compute
When building a custom LLM for summarization, which architecture is most suitable and why?
Answer: T5, because it treats summarization as a text-to-text task with a clear input-output format.. The correct answer is T5 because it is explicitly designed for text-to-text tasks like summarization, where the input (a long document) and output (a summary) are both treated as text. GPT is better for open-ended generation, BERT lacks generative capabilities, and hybrids are unnecessarily complex for this task.
From lesson: Common LLM Architectures: GPT, BERT, T5, and Their Variants
You are fine-tuning a pre-trained BERT model for a classification task with a small dataset. What is the best practice to avoid overfitting?
Answer: Freeze the early layers and only fine-tune the later layers with a low learning rate.. The correct answer is to freeze early layers and fine-tune later layers with a low learning rate. Early layers capture general language features, while later layers specialize in task-specific patterns. Freezing prevents overfitting to small datasets. High learning rates or custom tokenizers are unnecessary, and batch size alone doesn’t address overfitting.
From lesson: Common LLM Architectures: GPT, BERT, T5, and Their Variants
Why does GPT use a unidirectional attention mechanism, while BERT uses bidirectional attention?
Answer: GPT is designed for generative tasks where future tokens must not influence past tokens, while BERT is optimized for tasks requiring full context understanding.. The correct answer is that GPT’s unidirectional attention prevents future tokens from influencing past tokens (critical for generation), while BERT’s bidirectional attention captures full context (critical for tasks like classification). The other options misrepresent the trade-offs or are factually incorrect.
From lesson: Common LLM Architectures: GPT, BERT, T5, and Their Variants
You are implementing a custom LLM for a translation task. Which architecture and input format should you use?
Answer: T5 with a prefix like 'translate English to French: [text]'.. The correct answer is T5 with a prefix like 'translate English to French: [text]' because T5 is designed for text-to-text tasks and explicitly uses task prefixes. GPT can generate translations but lacks T5’s structured input-output format. BERT is not generative, and omitting positional embeddings would break the model.
From lesson: Common LLM Architectures: GPT, BERT, T5, and Their Variants
During fine-tuning, your custom LLM’s loss is oscillating wildly. What is the most likely cause and fix?
Answer: The batch size is too small; increase it to stabilize gradients.. The correct answer is that the batch size is too small, leading to noisy gradients. Increasing batch size stabilizes training. A low learning rate would cause slow convergence, not oscillation. Missing positional embeddings or tokenizer mismatches would cause consistent errors, not oscillations.
From lesson: Common LLM Architectures: GPT, BERT, T5, and Their Variants
During dataset preparation for your LLM, you notice that the text corpus contains more examples from one demographic group than others. What is the most effective way to address this imbalance while preserving the model's utility?
Answer: Use data augmentation to generate synthetic examples for underrepresented groups, while applying fairness-aware sampling to maintain balance during training.. The correct answer is using data augmentation and fairness-aware sampling. This approach preserves the utility of the model by maintaining sufficient data volume while actively mitigating bias. Option 1 harms utility by discarding data. Option 3 ignores the problem, risking biased outputs. Option 4 may not fully address biases and could degrade performance.
From lesson: Ethical Considerations: Bias, Fairness, and Misuse in LLMs
You are evaluating your LLM's fairness across different demographic groups. Which of the following metrics would best reveal disparities that might be hidden by overall accuracy?
Answer: Precision and recall for each demographic group separately, to identify where the model performs poorly.. The correct answer is precision and recall for each group separately. These metrics reveal disparities that aggregate accuracy might hide. Option 1 masks group-specific issues. Option 3 is incorrect because dataset size doesn't guarantee fairness. Option 4 confuses confidence with fairness, which are unrelated.
From lesson: Ethical Considerations: Bias, Fairness, and Misuse in LLMs
Your LLM is intended for use in a hiring tool. During testing, you find that the model favors resumes with certain keywords that are more common in one demographic group. What is the most responsible action to take before deployment?
Answer: Conduct a context-specific risk assessment, including red-teaming for potential misuse, and adjust the model or deployment constraints based on the findings.. The correct answer is conducting a context-specific risk assessment. This ensures the model's safety and fairness in its intended use case. Option 1 ignores the problem. Option 2 is impractical and may harm performance. Option 4 may not address the root cause of the bias and could introduce new issues.
From lesson: Ethical Considerations: Bias, Fairness, and Misuse in LLMs
You are designing an LLM and want to integrate fairness constraints into the training process. Which approach would most effectively reduce bias while maintaining model performance?
Answer: Add a fairness-aware regularization term to the loss function that penalizes disparities in performance across groups.. The correct answer is adding a fairness-aware regularization term to the loss function. This approach actively reduces bias during training without sacrificing performance. Option 1 is a post-hoc fix that may not address underlying biases. Option 2 is impractical and may exclude relevant data. Option 3 is unrealistic and could harm generalization.
From lesson: Ethical Considerations: Bias, Fairness, and Misuse in LLMs
After fine-tuning your LLM on a small, curated dataset to reduce biases, you observe that the model's performance has degraded for some groups. What is the most likely cause of this issue?
Answer: The curated dataset was too small to maintain the model's generalization capabilities, leading to overfitting on specific examples.. The correct answer is that the curated dataset was too small, leading to overfitting. Fine-tuning on a limited dataset can reduce generalization. Option 1 is incorrect because architecture isn't the issue. Option 2 is false; original data is rarely perfectly balanced. Option 3 is a misconception; fairness and performance can coexist with proper techniques.
From lesson: Ethical Considerations: Bias, Fairness, and Misuse in LLMs
Why is deduplication important when curating a dataset for training an LLM?
Answer: It prevents the model from overfitting to repeated patterns, improving generalization.. The correct answer is that deduplication prevents overfitting. Overfitting occurs when the model memorizes repeated patterns instead of learning generalizable rules. While deduplication may reduce storage and training time, these are secondary benefits. Copyright compliance requires separate checks.
From lesson: Data Collection and Curation for Training LLMs
A team scrapes web data for their LLM but skips license verification. What is the most critical risk?
Answer: The team could face legal action, forcing them to discard the dataset and retrain.. The correct answer is legal risk. Using data without proper licenses can lead to lawsuits or forced retraining, which is costly. While toxic content and slow training are concerns, they are not as critical as legal consequences. Metadata is rarely required for coherence.
From lesson: Data Collection and Curation for Training LLMs
How does balancing dataset size and quality impact an LLM's performance?
Answer: A smaller, high-quality dataset often generalizes better than a large, noisy one.. The correct answer is that a smaller, high-quality dataset often generalizes better. Noisy or biased data in a large dataset can degrade performance. While size matters, quality is equally critical. Architecture alone cannot compensate for poor data.
From lesson: Data Collection and Curation for Training LLMs
What is the primary purpose of documenting data sources and preprocessing steps?
Answer: To enable reproducibility and simplify debugging of model issues.. The correct answer is reproducibility and debugging. Documentation ensures others can replicate the work and helps identify data-related issues. While it may aid automation, the primary goal is transparency, not stakeholder reports or size reduction.
From lesson: Data Collection and Curation for Training LLMs
A dataset includes many samples with incorrect labels. How does this most likely affect the LLM?
Answer: The model's outputs will become unpredictable or nonsensical due to conflicting signals.. The correct answer is unpredictable outputs. Incorrect labels confuse the model, leading to poor performance. The model cannot ignore labels or correct them automatically. Training speed is unaffected by label quality.
From lesson: Data Collection and Curation for Training LLMs
Why is it problematic to normalize numerical features (e.g., scaling to [0, 1]) before tokenizing text for an LLM?
Answer: Tokenization requires raw text; normalized numbers lose semantic meaning and break token alignment.. The correct answer is that tokenization requires raw text, and normalized numbers (e.g., '0.75' → '0.7') lose semantic meaning, breaking token alignment. Other options are wrong because: 1) Numerical features can be useful (e.g., in tabular data); 2) Normalization doesn’t directly distort embeddings; 4) Normalization is irrelevant post-tokenization.
From lesson: Data Cleaning and Preprocessing Pipelines
A dataset contains mixed code and natural language. What preprocessing step is most critical to preserve LLM performance?
Answer: Using a code-specific tokenizer to handle syntax like brackets and operators.. The correct answer is using a code-specific tokenizer, as it preserves syntax (e.g., brackets, operators) critical for code understanding. Other options are wrong because: 0) Punctuation is essential for code; 2) Lowercasing loses case-sensitive information (e.g., variable names); 3) Whitespace is often semantically meaningful in code.
From lesson: Data Cleaning and Preprocessing Pipelines
During preprocessing, you notice 20% of samples have missing values in a key column. What’s the best approach for an LLM pipeline?
Answer: Use a placeholder token (e.g., '[MISSING]') to explicitly mark gaps for the LLM.. The correct answer is using a placeholder token, as it preserves data integrity and lets the LLM learn to handle missingness. Other options are wrong because: 0) Dropping samples loses data; 1) Mean imputation is inappropriate for text; 3) LLMs cannot reliably infer missing context.
From lesson: Data Cleaning and Preprocessing Pipelines
Why might a preprocessing pipeline that works for a small dataset fail when scaled to a large dataset for LLM training?
Answer: Memory constraints prevent loading the entire dataset at once for processing.. The correct answer is memory constraints, as large datasets often exceed available RAM. Other options are wrong because: 0) Preprocessing complexity isn’t tied to overfitting; 2) Tokenization rules should be consistent; 3) Normalization is computationally cheap.
From lesson: Data Cleaning and Preprocessing Pipelines
An LLM’s performance drops after updating the preprocessing pipeline. What’s the most likely cause?
Answer: The pipeline changes altered token distributions, breaking compatibility with pretrained embeddings.. The correct answer is altered token distributions, as LLMs rely on consistent tokenization. Other options are wrong because: 0) Data leakage is rare in preprocessing; 2) Architectures aren’t auto-updated; 3) Preprocessing critically impacts performance.
From lesson: Data Cleaning and Preprocessing Pipelines
When training a 7B-parameter LLM from scratch, which hardware factor is MOST critical to avoid bottlenecks?
Answer: GPU memory bandwidth to feed data to compute units. The right answer is GPU memory bandwidth. LLMs are memory-bound during training due to large parameter counts and attention mechanisms. High bandwidth (e.g., H100's 3TB/s) ensures compute units aren't starved for data. VRAM capacity (option 0) is necessary but not sufficient if bandwidth is low. CPU cores (option 2) and storage (option 3) are secondary to GPU performance for this task.
From lesson: Choosing the Right Hardware: GPUs, TPUs, and Cloud Resources
A team is fine-tuning a 13B-parameter LLM on a cloud platform. They notice training is 3x slower than expected. Which is the LEAST likely cause?
Answer: Using on-demand pricing instead of spot instances. The right answer is using TPUs instead of GPUs. TPUs are optimized for large-scale training and can outperform GPUs for homogeneous workloads like fine-tuning. The other options are likely causes: single-GPU (option 0) limits throughput, slow NVLink (option 1) bottlenecks multi-GPU training, and on-demand pricing (option 3) doesn't affect speed but is unrelated to performance.
From lesson: Choosing the Right Hardware: GPUs, TPUs, and Cloud Resources
For a research project prototyping a novel LLM architecture, why might GPUs be preferred over TPUs?
Answer: TPUs require manual memory management for custom operations. The right answer is TPUs require manual memory management for custom operations. TPUs are designed for TensorFlow and may need XLA compilation for custom ops, limiting flexibility. GPUs support dynamic frameworks like PyTorch (option 1 is wrong; TPUs support PyTorch via libraries). FP32 performance (option 0) is irrelevant for LLMs (FP16/BF16 is standard). Cost-efficiency (option 2) depends on scale, not prototyping.
From lesson: Choosing the Right Hardware: GPUs, TPUs, and Cloud Resources
A startup is deploying a 30B-parameter LLM for inference. They observe high latency despite using high-end GPUs. What is the MOST likely solution?
Answer: Use model quantization to reduce memory bandwidth pressure. The right answer is model quantization. Quantization (e.g., FP16/INT8) reduces memory bandwidth usage, directly addressing latency in memory-bound inference. TPUs (option 0) are not inherently better for inference and may not support all LLM ops. Larger batch sizes (option 2) increase throughput but worsen latency. CPUs (option 3) are slower for LLMs due to lack of parallelism.
From lesson: Choosing the Right Hardware: GPUs, TPUs, and Cloud Resources
When selecting cloud resources for distributed LLM training, which factor is MOST often underestimated?
Answer: The impact of network topology on gradient synchronization. The right answer is the impact of network topology on gradient synchronization. Slow inter-node connections (e.g., Ethernet vs. InfiniBand) can dominate training time in distributed setups. Data egress costs (option 0) are secondary to performance. GPU-optimized filesystems (option 2) matter less for training than inference. Preemptible instances (option 3) are a cost, not performance, factor.
From lesson: Choosing the Right Hardware: GPUs, TPUs, and Cloud Resources
Why do we divide the dot product of queries and keys by the square root of the head dimension (d_k)?
Answer: To prevent large dot product values from resulting in very small gradients during backpropagation. Scaling by sqrt(d_k) prevents the dot product from growing too large in magnitude, which would push the softmax function into its saturation region where gradients are near zero. Other options suggest normalization or complexity reduction, which are not the primary purpose of this scaling factor.
From lesson: Implementing a Transformer Model from Scratch (Using PyTorch or TensorFlow)
In a decoder-only transformer, why is the causal mask (look-ahead mask) essential during training?
Answer: To prevent the model from 'seeing' future tokens when predicting the current token. The causal mask ensures that the prediction for a position depends only on known past positions. Without it, the model would cheat by looking at future tokens. Birectional representation is used in encoders, not decoders, and masking does not improve speed or reduce parameters.
From lesson: Implementing a Transformer Model from Scratch (Using PyTorch or TensorFlow)
What is the primary function of the Feed-Forward Network (FFN) block in each transformer layer?
Answer: To process each token position independently and introduce non-linearity. The FFN is applied position-wise, allowing the model to project features into a higher-dimensional space and apply non-linear activations. It does not perform communication between positions (that is the role of attention) and does not store the dataset.
From lesson: Implementing a Transformer Model from Scratch (Using PyTorch or TensorFlow)
When implementing multi-head attention, how does increasing the number of heads affect the model?
Answer: It allows the model to attend to information from different representation subspaces simultaneously. Multi-head attention enables the model to focus on different aspects of the input sequence in parallel. While the dimension per head decreases, the total parameter count remains roughly the same because the total hidden dimension is preserved. It does not ignore dependencies.
From lesson: Implementing a Transformer Model from Scratch (Using PyTorch or TensorFlow)
Why is the Residual Connection (Add & Norm) critical in a deep transformer?
Answer: It allows gradients to propagate through the network more easily, preventing vanishing gradients. Residual connections provide a 'highway' for gradients to flow backward during training, which is essential for training very deep models. They are not substitutes for attention or embeddings, and they do not influence the size of the attention matrix.
From lesson: Implementing a Transformer Model from Scratch (Using PyTorch or TensorFlow)
Why is learning rate warmup often used in LLM training?
Answer: To prevent gradient explosions by gradually increasing the learning rate from a small value. The correct answer is that warmup prevents gradient explosions by starting with a small learning rate and gradually increasing it. Option 1 is wrong because warmup doesn’t reduce memory. Option 3 is incorrect because warmup doesn’t skip steps. Option 4 is false because warmup isn’t about memorization.
From lesson: Training Loops and Optimization Techniques (Adam, Learning Rate Scheduling)
What is the primary role of Adam’s beta2 parameter (e.g., 0.999) in LLM training?
Answer: It exponentially decays the moving average of squared gradients to estimate second moments. Beta2 in Adam decays the moving average of squared gradients (second moments) to adapt the learning rate per parameter. Option 1 describes beta1 (momentum). Option 3 is partially correct but misstates the purpose. Option 4 is unrelated to beta2.
From lesson: Training Loops and Optimization Techniques (Adam, Learning Rate Scheduling)
During LLM training, the loss suddenly spikes to NaN. Which of these is the LEAST likely cause?
Answer: The model’s weight initialization was too small (e.g., near zero). Small weight initialization (Option 2) typically causes slow convergence, not NaN spikes. The others (no clipping, high LR, low epsilon) are common causes of instability. Option 3 is the least likely.
From lesson: Training Loops and Optimization Techniques (Adam, Learning Rate Scheduling)
How does cosine learning rate decay benefit LLM training compared to step decay?
Answer: It smoothly decreases the learning rate to a minimum, avoiding abrupt changes that could destabilize training. Cosine decay smoothly reduces the learning rate, avoiding abrupt changes that could harm training. Option 1 describes step decay. Option 3 is incorrect because cosine decay decreases the rate. Option 4 is unrelated to cosine decay.
From lesson: Training Loops and Optimization Techniques (Adam, Learning Rate Scheduling)
An LLM’s training loss plateaus early. Which of these is the MOST likely fix?
Answer: Reduce the learning rate or introduce learning rate warmup. A plateau often indicates the learning rate is too high or lacks warmup. Reducing the rate or adding warmup helps. Option 1 may worsen plateaus. Option 3 (switching to SGD) is rarely used for LLMs. Option 4 would likely cause instability.
From lesson: Training Loops and Optimization Techniques (Adam, Learning Rate Scheduling)
When training a large language model that exceeds the memory of a single GPU, which combination of techniques is most effective?
Answer: Use model parallelism to split the model across GPUs and data parallelism to distribute the dataset.. The correct answer is combining model parallelism (to split the model) and data parallelism (to distribute the dataset). Option 1 fails because data parallelism alone cannot handle models larger than a single GPU's memory. Option 3 is impractical as it may exceed memory limits or degrade model quality. Option 4 reduces memory usage but doesn't address the core issue of model size.
From lesson: Distributed Training: Data Parallelism and Model Parallelism
In data parallelism, why is gradient synchronization a potential bottleneck?
Answer: Because all devices must communicate gradients to ensure consistent model updates, which scales poorly with more devices.. The correct answer is that all devices must communicate gradients (e.g., via all-reduce) to ensure consistent model updates, which becomes a bottleneck with more devices or slow interconnects. Option 1 is wrong because gradients are computed on all devices. Option 3 is incorrect because gradients are essential for backpropagation. Option 4 is wrong because synchronization is critical during training.
From lesson: Distributed Training: Data Parallelism and Model Parallelism
How does model parallelism differ from pipeline parallelism in distributed training?
Answer: Model parallelism splits the model's tensors or layers across devices, while pipeline parallelism splits the batch of data into micro-batches and pipelines execution across devices.. The correct answer is that model parallelism splits the model's tensors or layers across devices, while pipeline parallelism splits the batch into micro-batches and pipelines execution. Option 1 reverses the definitions. Option 3 is incorrect because they are distinct techniques. Option 4 is wrong because both techniques are used for training and inference.
From lesson: Distributed Training: Data Parallelism and Model Parallelism
What is a key challenge when implementing model parallelism for a transformer-based LLM?
Answer: Minimizing communication between devices when layers are split across them.. The correct answer is minimizing communication between devices when layers are split, as cross-device communication can introduce latency. Option 1 is wrong because model parallelism is used precisely when layers don't fit in a single GPU. Option 3 is incorrect because gradient synchronization is still required for training. Option 4 is unrelated to model parallelism's core challenge.
From lesson: Distributed Training: Data Parallelism and Model Parallelism
Why might increasing the number of GPUs in data parallelism lead to diminishing returns?
Answer: Because the overhead of gradient synchronization increases with more GPUs, outweighing the benefits of additional compute.. The correct answer is that the overhead of gradient synchronization (e.g., all-reduce operations) increases with more GPUs, eventually outweighing the benefits of additional compute. Option 1 is wrong because model size doesn't grow with GPUs in data parallelism. Option 3 is incorrect because data parallelism scales to many GPUs. Option 4 is misleading because the dataset is automatically split in frameworks like PyTorch or TensorFlow.
From lesson: Distributed Training: Data Parallelism and Model Parallelism
Which of the following scenarios best demonstrates why Perplexity might be misleading for an LLM intended for creative writing?
Answer: Perplexity penalizes diverse, highly creative word choices that deviate from the most statistically probable next tokens.. Perplexity is a probabilistic metric that favors the most likely next tokens; in creative writing, the 'best' word is often less probable. Option 0 is a specific failure mode, option 1 is false because fluency and probability differ, and option 3 is false because Perplexity cannot measure semantic intent.
From lesson: Evaluating Model Performance: Perplexity, BLEU, and Human Evaluation
When comparing two different model checkpoints, what is the primary limitation of using BLEU score as the sole metric?
Answer: BLEU lacks a mechanism to understand semantic equivalence when different words are used to convey the same meaning.. BLEU is an n-gram overlap metric; it treats synonyms or paraphrases as errors. Option 0 is false, option 2 is false as it is computationally light, and option 3 is false as it handles length via brevity penalties.
From lesson: Evaluating Model Performance: Perplexity, BLEU, and Human Evaluation
In a human evaluation study, why is it critical to provide raters with a clear rubric and a 'Gold Standard' example?
Answer: To reduce subjective variance and ensure raters apply consistent criteria for quality, such as fluency or accuracy.. Rubrics normalize human interpretation. Option 0 is irrelevant, option 2 is incorrect as humans should evaluate independently of flawed automated metrics, and option 3 would compromise the quality of the evaluation.
From lesson: Evaluating Model Performance: Perplexity, BLEU, and Human Evaluation
What does a significant decrease in Perplexity during the training of an LLM generally suggest about the model's status?
Answer: The model is becoming better at assigning higher probability to the ground-truth tokens in the validation set.. Perplexity is essentially the exponentiated cross-entropy; a decrease directly maps to better prediction of the next token. Options 0, 2, and 3 are assumptions not supported by the math of Perplexity.
From lesson: Evaluating Model Performance: Perplexity, BLEU, and Human Evaluation
If an LLM has a very low Perplexity but receives poor scores in a human evaluation of its helpfulness, what is the most likely issue?
Answer: The model is optimized to predict likely sequences based on the training data but fails to align with the specific intent or instructions of the human prompt.. This highlights the 'Alignment' problem: a model can be statistically accurate (low perplexity) but unhelpful. Option 0 is an excuse, option 2 is irrelevant to evaluation, and option 3 is a theoretical contradiction.
From lesson: Evaluating Model Performance: Perplexity, BLEU, and Human Evaluation
Why is it important to use a lower learning rate for earlier layers when fine-tuning a pre-trained model?
Answer: Earlier layers contain general features that are already well-optimized, and a high learning rate could disrupt them.. The correct answer is that earlier layers contain general features (e.g., edges, patterns) that are already well-optimized during pre-training. A high learning rate could disrupt these features, harming performance. The other options are incorrect because: earlier layers are not randomly initialized, lower learning rates are not exclusive to the final layer, and learning rates do affect unfrozen layers.
From lesson: Fine-Tuning Pre-Trained Models for Specific Tasks
A practitioner fine-tunes a pre-trained model on a small dataset and observes poor performance on unseen data. What is the most likely cause and fix?
Answer: The model is overfitting; use data augmentation or regularization techniques like dropout.. The correct answer is that the model is likely overfitting due to the small dataset. Data augmentation or regularization (e.g., dropout) can help improve generalization. The other options are incorrect because: increasing the learning rate may worsen overfitting, tokenizer size isn't the primary issue, and batch size has minimal impact on overfitting.
From lesson: Fine-Tuning Pre-Trained Models for Specific Tasks
When fine-tuning a pre-trained model for a task with domain-specific terms, what is the best approach to handle tokenization?
Answer: Extend the pre-trained tokenizer's vocabulary with domain-specific tokens while keeping the original embeddings.. The correct answer is to extend the pre-trained tokenizer's vocabulary with domain-specific tokens. This leverages the existing tokenizer's strengths while adapting to new terms. The other options are incorrect because: using the tokenizer as-is may poorly handle domain terms, training a new tokenizer from scratch loses pre-trained knowledge, and ignoring tokenization harms performance.
From lesson: Fine-Tuning Pre-Trained Models for Specific Tasks
What is the primary risk of fine-tuning all layers of a pre-trained model with a high learning rate?
Answer: The model may lose its pre-trained knowledge and fail to generalize to the new task.. The correct answer is that the model may lose its pre-trained knowledge (catastrophic forgetting) and fail to generalize. High learning rates can disrupt the well-optimized weights of pre-trained layers. The other options are incorrect because: quick convergence isn't the primary risk, computational cost isn't the main concern, and interpretability isn't directly impacted by learning rate.
From lesson: Fine-Tuning Pre-Trained Models for Specific Tasks
A practitioner evaluates their fine-tuned model using only training loss and observes low loss values. Why might this not indicate good real-world performance?
Answer: Low training loss may indicate overfitting, where the model memorizes the training data but fails on unseen data.. The correct answer is that low training loss may indicate overfitting, where the model performs well on training data but poorly on unseen data. The other options are incorrect because: training loss can be lower than validation loss, perplexity is related to loss but not the only metric, and low training loss doesn't inherently imply architectural flaws.
From lesson: Fine-Tuning Pre-Trained Models for Specific Tasks
When quantizing an LLM from FP32 to INT8, you observe a 5% drop in accuracy on a summarization task. Which approach is most likely to recover the lost performance while maintaining compression?
Answer: Apply mixed-precision quantization, keeping sensitive layers (e.g., attention) in FP16 while quantizing others to INT8. The correct answer is mixed-precision quantization because it balances compression and accuracy by preserving precision in critical layers. Option 1 negates compression entirely. Option 3 increases model size, violating the goal of compression. Option 4 ignores the need for calibration, which is essential for minimizing quantization error.
From lesson: Model Compression: Quantization, Pruning, and Knowledge Distillation
You prune 30% of the weights in an LLM using magnitude-based pruning and observe a significant accuracy drop. What is the most effective way to mitigate this without increasing model size?
Answer: Use structured pruning (e.g., entire attention heads) followed by fine-tuning. Structured pruning followed by fine-tuning preserves model integrity by removing entire components (e.g., attention heads) rather than individual weights, making it easier to recover accuracy. Option 1 reduces compression effectiveness. Option 3 is computationally expensive and unnecessary. Option 4 may help but is less direct than fine-tuning the pruned structure.
From lesson: Model Compression: Quantization, Pruning, and Knowledge Distillation
In knowledge distillation for LLMs, why is it important to use a temperature-scaled softmax on the teacher model's logits?
Answer: To smooth the teacher's probability distribution, exposing the student to richer information about class relationships. Temperature scaling smooths the teacher's probability distribution, revealing relationships between classes (e.g., 'cat' vs. 'dog') that are otherwise obscured in hard labels. Option 1 is incorrect because temperature scaling increases computational cost. Option 2 describes the opposite effect (temperature actually reduces differences). Option 4 is unrelated to logit scaling.
From lesson: Model Compression: Quantization, Pruning, and Knowledge Distillation
You compress an LLM using 4-bit quantization and observe that its performance degrades significantly on a question-answering task but remains stable on text generation. What is the most likely explanation?
Answer: QA tasks rely more on precise attention mechanisms, which are sensitive to low-bit quantization. QA tasks often require precise attention to context and factual details, which are disrupted by aggressive quantization. Option 0 is unlikely because quantization noise is systematic, not random. Option 2 is incorrect because calibration data bias would affect both tasks. Option 3 is overly absolute.
From lesson: Model Compression: Quantization, Pruning, and Knowledge Distillation
When combining pruning and knowledge distillation to compress an LLM, which order of operations is most effective?
Answer: Prune first, then apply knowledge distillation to the pruned model. Pruning first reduces the model size, and distillation then helps recover accuracy by leveraging the teacher's knowledge. Option 1 risks pruning a model already compressed by distillation, compounding accuracy loss. Option 2 is less efficient because simultaneous training is complex and may not converge. Option 3 is incorrect because the techniques interact; pruning affects the student's capacity to learn from distillation.
From lesson: Model Compression: Quantization, Pruning, and Knowledge Distillation
Why is dynamic batching critical for serving LLMs efficiently?
Answer: It groups multiple inference requests to maximize GPU utilization and throughput.. The correct answer is that dynamic batching groups requests to maximize GPU utilization. The other options are incorrect: weight compression (option 0) is unrelated to batching, architecture conversion (option 2) is model distillation, and caching (option 3) is a separate optimization.
From lesson: Serving LLMs: APIs, Microservices, and Scalable Inference
What is the primary trade-off when quantizing an LLM for production?
Answer: Lower accuracy but reduced computational cost.. The correct answer is that quantization reduces computational cost (e.g., memory, compute) at the potential expense of accuracy. The other options are incorrect: quantization reduces memory usage (option 0), doesn't affect training time (option 2), and reduces model size (option 3).
From lesson: Serving LLMs: APIs, Microservices, and Scalable Inference
How do microservices improve the scalability of LLM serving?
Answer: By allowing independent scaling of components like tokenization and inference.. The correct answer is that microservices enable independent scaling of components. The other options are incorrect: combining components (option 0) is the opposite of microservices, reducing parameters (option 2) is model pruning, and caching (option 3) is unrelated to microservices.
From lesson: Serving LLMs: APIs, Microservices, and Scalable Inference
What is a key challenge of serving LLMs in a serverless environment?
Answer: Cold-start latency due to model loading delays.. The correct answer is cold-start latency, as serverless functions may spin down and reload the model. The other options are incorrect: serverless can handle batches (option 1), costs vary (option 2), and GPUs are supported (option 3).
From lesson: Serving LLMs: APIs, Microservices, and Scalable Inference
Why is per-token latency monitoring important for LLM serving?
Answer: It reveals inefficiencies in specific stages of the inference pipeline.. The correct answer is that per-token latency monitoring reveals inefficiencies in stages like token generation. The other options are incorrect: token order (option 0) is unrelated to latency, reducing tokens (option 2) is not a goal, and correctness (option 3) is not guaranteed by monitoring.
From lesson: Serving LLMs: APIs, Microservices, and Scalable Inference
When optimizing an LLM for inference, why might you choose TensorRT over ONNX Runtime?
Answer: TensorRT performs layer fusion and kernel auto-tuning for NVIDIA GPUs, improving throughput.. Correct: TensorRT is optimized for NVIDIA GPUs, offering layer fusion (e.g., merging convolution and ReLU) and kernel auto-tuning to maximize performance. Incorrect: Option 1 is wrong because ONNX Runtime supports more architectures, not TensorRT. Option 2 is backwards (TensorRT is NVIDIA-specific). Option 4 is incorrect because both tools support automatic quantization.
From lesson: Optimizing Inference: ONNX, TensorRT, and Model Optimization Tools
Your LLM’s inference speed is slow after converting to ONNX. What is the most likely cause?
Answer: The model was trained with unsupported ops (e.g., dynamic control flow) that ONNX cannot optimize.. Correct: ONNX may fail to optimize models with unsupported ops (e.g., dynamic loops), leading to suboptimal performance. Incorrect: Option 2 is false (ONNX Runtime supports GPUs). Option 3 is misleading (batch size affects throughput, not conversion). Option 4 is false (ONNX can match or exceed native performance).
From lesson: Optimizing Inference: ONNX, TensorRT, and Model Optimization Tools
How does quantization improve LLM inference, and what is a key trade-off?
Answer: It speeds up inference by using lower-precision data types (e.g., INT8), but can degrade output quality if applied too aggressively.. Correct: Quantization (e.g., INT8) reduces memory bandwidth and compute requirements, speeding up inference, but aggressive quantization can hurt accuracy. Incorrect: Option 1 is wrong (quantization typically reduces latency). Option 3 is partially true but not the primary trade-off. Option 4 is false (quantization works with other tools and still benefits from GPUs).
From lesson: Optimizing Inference: ONNX, TensorRT, and Model Optimization Tools
You observe that your LLM’s outputs are less coherent after applying TensorRT optimizations. What should you investigate first?
Answer: Whether the quantization calibration dataset was representative of the LLM’s input distribution.. Correct: Poor quantization calibration (e.g., using a non-representative dataset) can degrade accuracy. Incorrect: Option 1 is false (TensorRT supports mixed precision). Option 3 is unlikely (layer fusion doesn’t modify attention logic). Option 4 is irrelevant (vocabulary size isn’t affected by TensorRT).
From lesson: Optimizing Inference: ONNX, TensorRT, and Model Optimization Tools
Why might you use ONNX Runtime instead of TensorRT for deploying an LLM?
Answer: ONNX Runtime supports more hardware platforms (e.g., CPUs, AMD GPUs) and frameworks (e.g., PyTorch, TensorFlow).. Correct: ONNX Runtime is hardware-agnostic and supports more frameworks, making it versatile for cross-platform deployment. Incorrect: Option 2 is backwards (TensorRT handles fusion automatically). Option 3 is false (ONNX Runtime supports GPUs). Option 4 is false (accuracy depends on optimization settings, not the tool itself).
From lesson: Optimizing Inference: ONNX, TensorRT, and Model Optimization Tools
When building a user interface for your LLM, why might you choose a custom frontend over Gradio or Streamlit?
Answer: A custom frontend offers better scalability, security, and customization for production deployments with high traffic or complex workflows.. The correct answer is that a custom frontend offers better scalability, security, and customization. Gradio and Streamlit are great for prototyping but may struggle with high traffic or complex requirements. The other options are incorrect: Gradio/Streamlit can integrate with LLMs (option 1 and 3 are false), and they are often easier to develop for simple use cases (option 4 is false).
From lesson: Building a User Interface for Your LLM: Gradio, Streamlit, or Custom Frontend
What is a critical security risk of embedding API keys directly in a Gradio or Streamlit frontend?
Answer: API keys exposed in client-side code can be easily extracted by users, leading to unauthorized access or abuse.. The correct answer is that API keys exposed in client-side code can be extracted by users. This is a major security risk, as attackers could abuse the key or access sensitive data. The other options are incorrect: API keys are not encrypted in client-side code (option 1), key length/special characters don't cause crashes (option 2), and Gradio/Streamlit will run but remain insecure (option 4).
From lesson: Building a User Interface for Your LLM: Gradio, Streamlit, or Custom Frontend
Why is input validation important in a user interface for an LLM?
Answer: It prevents malicious or malformed inputs from crashing the backend, exposing vulnerabilities, or degrading performance.. The correct answer is that input validation prevents malicious or malformed inputs from causing issues. LLMs are not immune to bad inputs, which can crash the backend or expose security flaws. The other options are incorrect: LLMs can fail with bad inputs (option 1), validation is needed for all frontends (option 3), and validation doesn't guarantee correct LLM responses (option 4).
From lesson: Building a User Interface for Your LLM: Gradio, Streamlit, or Custom Frontend
What is a key advantage of using Gradio or Streamlit for prototyping an LLM interface?
Answer: They provide built-in tools for rapid development, deployment, and sharing of LLM interfaces with minimal setup.. The correct answer is that Gradio and Streamlit provide built-in tools for rapid development and deployment. They simplify the process of creating and sharing LLM interfaces. The other options are incorrect: they do require coding (option 1), they don't optimize LLM performance (option 3), and they are not the only tools for LLM integration (option 4).
From lesson: Building a User Interface for Your LLM: Gradio, Streamlit, or Custom Frontend
How can you improve the user experience of an LLM interface when dealing with long response times?
Answer: Add loading indicators, stream responses incrementally, and provide feedback to keep users informed during delays.. The correct answer is to add loading indicators, stream responses, and provide feedback. This keeps users engaged and informed during delays. The other options are incorrect: disabling input frustrates users (option 1), temperature doesn't control response length/speed (option 3), and sacrificing quality for speed is not ideal (option 4).
From lesson: Building a User Interface for Your LLM: Gradio, Streamlit, or Custom Frontend
When monitoring an LLM in production, why is 'drift detection' particularly challenging compared to traditional software?
Answer: Semantic drift is difficult to quantify because input data is unstructured and contextual.. Semantic drift is hard to measure because language is ambiguous; traditional metrics fail here. Option 0 is false as architectures are static. Option 2 is false as monitoring is vital. Option 3 is false as LLMs have version control.
From lesson: Monitoring and Maintaining LLMs in Production
What is the primary purpose of implementing an evaluation loop using 'LLM-as-a-judge'?
Answer: To provide a scalable way to evaluate responses against specific rubrics or grounding criteria.. LLM-as-a-judge uses a robust model to score outputs against specific criteria, scaling evaluation. Option 0 is dangerous, Option 1 is technically incorrect, and Option 3 is too narrow.
From lesson: Monitoring and Maintaining LLMs in Production
In a production environment, why should you prioritize tracking the 'average tokens per request'?
Answer: It helps identify prompt injection attempts that consume excessive resources.. Sudden spikes in tokens often indicate complex prompts or malicious injections. Options 0, 2, and 3 are incorrect as they do not reflect the operational impact of token usage.
From lesson: Monitoring and Maintaining LLMs in Production
How does 'Golden Dataset' maintenance improve the reliability of a production LLM?
Answer: It allows for regression testing to ensure updates do not degrade performance on known critical tasks.. A Golden Dataset provides a baseline for regression testing. Option 0 describes training, not evaluation. Option 2 is impossible in LLMs, and Option 3 ignores the need for human oversight.
From lesson: Monitoring and Maintaining LLMs in Production
What is the primary risk of relying on a high 'semantic similarity' score for production evaluation?
Answer: Similarity metrics do not guarantee factual correctness or logical consistency.. Similarity scores measure wording, not truth or logic. Option 0 is irrelevant to logic, Option 2 is false, and Option 3 is incorrect as these metrics are widely supported.
From lesson: Monitoring and Maintaining LLMs in Production
When training a reward model in RLHF, what is the primary purpose of using a pairwise comparison dataset?
Answer: To approximate the latent human preference function that is difficult to define mathematically.. Pairwise comparisons allow us to train a scalar reward function that maps responses to preference scores, which is essential because human 'goodness' is subjective and not easily expressed as a hard loss function. SFT does not use pairwise data, and this process does not replace pre-training or affect tokenization.
From lesson: Advanced Topics: Reinforcement Learning from Human Feedback (RLHF)
What happens if the KL divergence penalty in PPO is set to zero during RLHF?
Answer: The policy will likely collapse by generating repetitive or 'gibberish' text that exploits the reward model.. Without the KL penalty, the RL algorithm will find extreme, nonsensical text sequences that trigger the highest reward scores in the model, a process known as reward hacking. The KL penalty forces the model to stay close to its pre-trained state, preventing it from deviating into non-linguistic output.
From lesson: Advanced Topics: Reinforcement Learning from Human Feedback (RLHF)
Which of the following describes the fundamental challenge of 'Reward Hacking' in RLHF?
Answer: The model finds unintended shortcuts that satisfy the reward model without genuinely following human instructions.. Reward hacking occurs when the model finds a loophole in the reward model's logic. Option 1 is a technical limitation, not a core RLHF alignment challenge. Option 3 is a data quality issue, and Option 4 is an infrastructure constraint, not a logic-based failure of RLHF.
From lesson: Advanced Topics: Reinforcement Learning from Human Feedback (RLHF)
Why is it often preferred to use a separate Reward Model instead of using human feedback directly during the reinforcement learning step?
Answer: Human feedback is too slow and expensive to provide in real-time during the agent's gradient update loops.. RL requires thousands of interactions; humans cannot provide feedback at that speed. Option 1 is correct. Human feedback is notoriously inconsistent (Option 2), RLHF does not replace pre-training (Option 3), and RLHF often amplifies rather than eliminates biases (Option 4).
From lesson: Advanced Topics: Reinforcement Learning from Human Feedback (RLHF)
In the context of RLHF, what does 'sycophancy' refer to?
Answer: The model providing factually incorrect information to please the user's incorrect premise.. Sycophancy in RLHF occurs when a model learns that 'being helpful' translates to agreeing with the user, even if the user is wrong. This happens when the reward model is trained on preferences that prioritize being agreeable over being accurate. The other options describe repetition, data cleaning, and ensemble methods, none of which define sycophancy.
From lesson: Advanced Topics: Reinforcement Learning from Human Feedback (RLHF)
When training a model that exceeds the memory of a single GPU, which feature of DeepSpeed is primarily responsible for partitioning model parameters, gradients, and optimizer states across multiple devices?
Answer: ZeRO Redundancy Optimizer. ZeRO (Zero Redundancy Optimizer) specifically targets memory savings by partitioning optimizer states and parameters across GPUs. Pipeline parallelism splits layers across devices rather than parameters; Gradient Checkpointing saves activation memory; LoRA is a fine-tuning technique, not a memory-partitioning framework feature.
From lesson: Exploring Open-Source LLM Frameworks: Hugging Face Transformers, DeepSpeed, and Megatron-LM
What is the primary architectural trade-off when using Megatron-LM for Tensor Parallelism on an LLM?
Answer: Increased communication overhead due to all-reduce operations. Tensor parallelism requires splitting operations across devices, necessitating frequent all-reduce calls between GPUs, which creates communication overhead. It does not simplify data pipelines or reduce interconnect needs; if anything, it demands high-speed interconnects to maintain efficiency.
From lesson: Exploring Open-Source LLM Frameworks: Hugging Face Transformers, DeepSpeed, and Megatron-LM
In the context of the Hugging Face Trainer, why is it recommended to use 'fp16' or 'bf16' for large language models?
Answer: To reduce memory usage and speed up compute on modern hardware. Mixed precision training reduces the memory footprint of weights and activations, and modern GPUs accelerate half-precision operations significantly. This does not change model intelligence, eliminate scaling needs (it often requires them), or replace the optimizer function.
From lesson: Exploring Open-Source LLM Frameworks: Hugging Face Transformers, DeepSpeed, and Megatron-LM
Why is Pipeline Parallelism often utilized in tandem with Data Parallelism when scaling to thousands of GPUs?
Answer: To increase the effective batch size while managing memory per layer. Pipeline parallelism allows deep models to be split vertically, while data parallelism scales throughput. This combination increases the effective batch size. It does not replace ZeRO, change convergence rates, or eliminate the need for DDP.
From lesson: Exploring Open-Source LLM Frameworks: Hugging Face Transformers, DeepSpeed, and Megatron-LM
If your training job is bottlenecked by CPU-to-GPU data transfer when using Hugging Face datasets, what is the most effective architectural adjustment?
Answer: Increasing the num_workers in the data loader and using prefetching. Increasing num_workers allows for parallel data fetching, which hides the latency of data preparation. A larger model would exacerbate the compute demand, a lower batch size underutilizes the hardware, and changing the optimizer does not address the data pipeline bottleneck.
From lesson: Exploring Open-Source LLM Frameworks: Hugging Face Transformers, DeepSpeed, and Megatron-LM
Why is the Scaled Dot-Product Attention mechanism divided by the square root of the dimension of the key vectors (sqrt(d_k))?
Answer: To prevent the dot product from growing too large in magnitude, which pushes gradients into regions with tiny derivatives.. Dividing by sqrt(d_k) prevents extremely large dot products that force the softmax into a 'saturated' state where gradients vanish. Option 0 is a side effect, not the primary reason; option 2 happens regardless of scaling; option 3 is false as scaling does not change the summation requirement.
From lesson: Explain the transformer architecture and its key components.
What is the primary purpose of the residual connections around the attention and feed-forward sub-layers?
Answer: To allow the model to learn identity mappings, which facilitates the training of very deep architectures.. Residual connections provide a 'shortcut' for gradients to flow through the network, preventing vanishing gradients in deep stacks. The other options are incorrect as they do not describe the fundamental function of residual blocks in deep learning.
From lesson: Explain the transformer architecture and its key components.
In a decoder-only model, why is masking applied to the input during self-attention?
Answer: To ensure that during training, a position cannot attend to tokens that appear after it in the sequence.. Masking enforces the causal constraint required for autoregressive generation, ensuring the model only uses past info. Option 0 is irrelevant to masking; option 1 describes the encoder's task; option 3 is the purpose of masking.
From lesson: Explain the transformer architecture and its key components.
How does Multi-Head Attention contribute to the transformer's performance compared to a single attention head?
Answer: It allows the model to simultaneously attend to information from different representation subspaces at different positions.. Multiple heads allow the model to focus on different aspects of a sequence (e.g., syntax, semantics) concurrently. Option 1 is false because heads add overhead; option 2 is incorrect as positional encoding remains necessary; option 3 is false as heads have separate weights.
From lesson: Explain the transformer architecture and its key components.
Why is the feed-forward network (FFN) applied to each position separately and identically in the transformer block?
Answer: To allow for shared computation and parameter efficiency across all tokens in the sequence.. The position-wise FFN allows the model to process tokens independently, making it highly efficient for parallelization. Option 0 is not the primary reason; options 1 and 3 do not capture the efficiency/independence rationale.
From lesson: Explain the transformer architecture and its key components.
When training your own model, why is it insufficient to simply remove explicit sensitive terms from the training data?
Answer: Models can infer demographic associations through non-sensitive, correlated features in the data.. Correct: Models learn patterns and correlations; removing explicit labels does not remove the underlying structural bias. Incorrect: Removing terms does not increase training time, impact syntax, or trigger safety layers.
From lesson: How do you handle bias and fairness in large language models?
Which of the following describes the benefit of adversarial fine-tuning for fairness?
Answer: It intentionally exposes the model to edge cases to identify and suppress biased reasoning patterns.. Correct: Adversarial training probes weaknesses to build robustness against bias. Incorrect: The other options describe general language modeling or filtering, which do not address the active mitigation of biased reasoning.
From lesson: How do you handle bias and fairness in large language models?
If your model shows a disparate impact across demographic groups, which intervention is most effective during the alignment phase?
Answer: Adding a regularization penalty for biased outputs during Reinforcement Learning from Human Feedback.. Correct: Integrating fairness as a reward signal allows the model to optimize for both performance and fairness. Incorrect: The other choices are either ineffective at addressing semantic bias or harmful to general model capability.
From lesson: How do you handle bias and fairness in large language models?
Why is 'Fairness' often considered a multi-objective optimization problem in custom model development?
Answer: It requires balancing model accuracy against the desire to remove specific societal biases.. Correct: There is often a trade-off between strict fairness constraints and raw model accuracy. Incorrect: The other options involve irrelevant technical constraints like languages or hardware.
From lesson: How do you handle bias and fairness in large language models?
What is the primary role of a 'Red Teaming' exercise in the model development cycle?
Answer: To simulate attacks or queries designed to elicit biased or harmful behavior from the model.. Correct: Red teaming is specifically designed to discover failures in safety and bias. Incorrect: Speed, summarization, and self-re-training are not the primary purposes of fairness-focused red teaming.
From lesson: How do you handle bias and fairness in large language models?
When training a model across multiple nodes, why is ZeRO (Zero Redundancy Optimizer) preferred over standard distributed training?
Answer: It eliminates redundant storage of optimizer states and gradients across GPUs.. ZeRO is designed to eliminate memory redundancy by sharding optimizer states across GPUs. Option 0 is wrong because ZeRO doesn't use disk; Option 2 is wrong because ZeRO is typically used within data parallelism; Option 3 is wrong because it does not handle hyperparameter tuning.
From lesson: What are the challenges of training large language models at scale?
What is the primary risk of using an excessively large global batch size without proper tuning?
Answer: The model may fail to converge as the optimizer step size becomes too aggressive for the data distribution.. Large batches can cause the model to land in sharp minima, leading to poor generalization. Option 0 is irrelevant to batch size; Option 1 is wrong because larger batches usually smoothen gradients; Option 3 is a network infrastructure concern, not a primary convergence risk.
From lesson: What are the challenges of training large language models at scale?
Why is mixed-precision training (FP16 or BF16) essential for scaling large language models?
Answer: It significantly reduces GPU memory usage and increases throughput without sacrificing meaningful convergence.. Mixed-precision reduces memory footprint and leverages hardware tensor cores for faster arithmetic. Option 0 is wrong as layers aren't removed; Option 2 is wrong because noise injection is not the purpose; Option 3 is wrong because throughput is independent of interconnect protocols.
From lesson: What are the challenges of training large language models at scale?
In a distributed environment, why might you observe lower than expected scaling efficiency despite using high-speed interconnects?
Answer: The overhead of synchronizing gradients across many nodes exceeds the computation time.. Communication overhead is a classic scaling limit. Option 1 describes a data loading bottleneck, which is a different issue; Option 2 is illogical; Option 3 is factually incorrect as optimizer update frequency is tied to the forward/backward pass cycles.
From lesson: What are the challenges of training large language models at scale?
How does Pipeline Parallelism address the limitations of training massive models on limited-VRAM devices?
Answer: It divides the model layers across different GPUs, allowing each to hold only a portion of the model.. Pipeline parallelism allows layering large models vertically across multiple devices. Option 1 describes data parallelism; Option 2 is slow due to RAM/GPU transfer latency; Option 3 is impractical due to storage I/O speeds.
From lesson: What are the challenges of training large language models at scale?
When fine-tuning a model, why do we typically prefer Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA over full-parameter fine-tuning?
Answer: It significantly reduces computational memory requirements by freezing most original weights. Option 1 is correct because PEFT reduces GPU memory footprint by updating only a small subset of weights. Option 0 is wrong because PEFT still requires a base model. Option 2 is wrong because fine-tuning doesn't grant fundamental new capabilities like entirely new linguistics. Option 3 is wrong because overfitting is still possible with PEFT if the dataset is poor.
From lesson: Describe the process of fine-tuning a pre-trained language model.
What is the primary risk of using a learning rate that is too high during the fine-tuning process?
Answer: Catastrophic forgetting of the general knowledge gained during pre-training. Option 2 is correct because high learning rates overwrite pre-trained weights, causing the model to lose original knowledge. Option 0 is wrong as high rates usually lead to instability rather than slow convergence. Option 1 is nonsensical, and Option 3 is unlikely since the model can still generate, but the quality will degrade significantly.
From lesson: Describe the process of fine-tuning a pre-trained language model.
How should you evaluate a fine-tuned model to ensure it is actually performing better for your specific task?
Answer: Compare performance on a held-out test set using both quantitative metrics and qualitative human review. Option 1 is correct as both quantitative metrics and human review are essential for robust evaluation. Option 0 is wrong because zero training loss indicates severe overfitting. Option 2 is wrong because training accuracy is a poor indicator of generalization. Option 3 is wrong because testing on training data does not measure the ability to handle unseen inputs.
From lesson: Describe the process of fine-tuning a pre-trained language model.
Why is it important to keep the input format consistent between fine-tuning and the final application?
Answer: To ensure the model recognizes the patterns and delimiters it learned during training. Option 1 is correct; models associate specific behaviors with specific tokens/structures used during training. Options 0, 2, and 3 are wrong because format consistency is about model behavior, not computational resource optimization.
From lesson: Describe the process of fine-tuning a pre-trained language model.
In the context of supervised fine-tuning (SFT), what is the main purpose of the dataset?
Answer: To map specific instructions or queries to high-quality desired outputs. Option 2 is correct; SFT involves learning from examples of input-response pairs. Option 0 is wrong because that describes pre-training, not SFT. Option 1 is wrong because next-word prediction is the pre-training objective. Option 3 is wrong because fine-tuning changes weights, not the number of parameters.
From lesson: Describe the process of fine-tuning a pre-trained language model.
When serving a large model, why is it preferred to use a specialized inference server rather than a standard REST framework like FastAPI?
Answer: They offer advanced features like continuous batching and KV-cache management that boost throughput. Inference servers manage GPU memory and request queues far more efficiently than standard web frameworks. Option 0 is false, as code architecture remains the same. Option 2 is false, as web frameworks handle JSON fine. Option 3 is false, as web frameworks are excellent at security.
From lesson: How would you deploy a large language model in a production environment?
What is the primary benefit of deploying a model with 4-bit quantization in a production environment?
Answer: It significantly reduces GPU memory usage, allowing larger models to fit on smaller instances. Quantization reduces bit precision, saving VRAM. Option 0 is false, as it may slightly reduce performance. Option 1 is false, as it's an optimization, not a device-locking mechanism. Option 3 is false, as quantization does not replace the API layer.
From lesson: How would you deploy a large language model in a production environment?
Why is 'continuous batching' superior to traditional static request batching for LLM inference?
Answer: It allows new requests to be added to a batch as soon as a sequence finishes, rather than waiting for the entire batch to complete. Continuous batching maximizes GPU utilization by filling slots as they become free. Option 1 is incorrect, as it doesn't affect token counts. Option 2 is false. Option 3 is false, as it is a scheduling mechanism, not a power management one.
From lesson: How would you deploy a large language model in a production environment?
When monitoring an LLM in production, which metric is most critical for evaluating user experience in real-time?
Answer: Time To First Token (TTFT) and throughput. Users care about how quickly the chat begins and how fast it generates text. Options 0, 1, and 3 are infrastructure metrics that do not directly measure the user's interactive experience.
From lesson: How would you deploy a large language model in a production environment?
What is the main challenge of scaling an LLM horizontally across multiple GPU nodes?
Answer: Network latency for tensor parallelism and cross-node communication can become a bottleneck. Splitting a model across nodes requires fast interconnects to communicate hidden states. Option 0 is false, as storage is rarely the bottleneck. Option 2 is irrelevant. Option 3 is false, as distributed inference is a standard practice.
From lesson: How would you deploy a large language model in a production environment?
