Building and Training Your Own LLM
Evaluating Model Performance: Perplexity, BLEU, and Human Evaluation
This lesson covers the primary methods used to measure LLM efficacy, spanning automated mathematical metrics and subjective qualitative assessments. Understanding these tools is essential to iterate on model architecture and data quality effectively. You will use these techniques whenever you need to compare baseline performance against refined versions of your model.
The Foundation: Understanding Perplexity
Perplexity is the most fundamental metric in language modeling, representing how well a probability model predicts a sample. Mathematically, it is the exponentiated average negative log-likelihood of a sequence. If a model has a perplexity of 10, it is as confused as if it were choosing uniformly at random between 10 words. Lower perplexity indicates that the model is more confident in assigning high probability to the ground truth next token, suggesting better internal representation of the data distribution. However, perplexity is not a measure of truthfulness or utility, but purely of predictive uncertainty. It effectively measures how well the model has compressed the training corpus; a lower score implies a more accurate representation of the underlying linguistic patterns, making it the primary objective during initial training phases to ensure the weights are converging toward a useful state.
import torch
def calculate_perplexity(model, data_loader):
# Set model to evaluation mode
model.eval()
total_loss = 0
total_tokens = 0
with torch.no_grad():
for batch in data_loader:
logits = model(batch['input_ids'])
# Calculate cross entropy loss across the sequence
loss = torch.nn.functional.cross_entropy(logits.view(-1, logits.size(-1)), batch['labels'].view(-1))
total_loss += loss.item() * batch['input_ids'].numel()
total_tokens += batch['input_ids'].numel()
# Exponentiate the average loss to get perplexity
return torch.exp(torch.tensor(total_loss / total_tokens))Assessing Translation and Paraphrasing: BLEU Score
The BLEU (Bilingual Evaluation Understudy) score measures the overlap between a machine-generated sequence and a human-provided reference text using n-gram precision. It works by calculating the percentage of n-grams that appear in the candidate translation that are also present in the reference, adjusted for brevity through a penalty factor. Because it relies purely on token matching, it is highly efficient for automated testing in environments where training pipelines are frequent. However, BLEU is inherently limited because it does not account for semantic meaning, synonyms, or contextual nuance. A sentence could have a perfect meaning but a zero BLEU score if it uses different vocabulary than the reference. It is best used for benchmarking factual extraction or rigid tasks where precision in keyword retention is a mandatory requirement for performance assessment during the development lifecycle.
from collections import Counter
def calculate_bleu(candidate, reference, n=1):
# Count n-grams in candidate and reference
cand_grams = Counter([tuple(candidate[i:i+n]) for i in range(len(candidate)-n+1)])
ref_grams = Counter([tuple(reference[i:i+n]) for i in range(len(reference)-n+1)])
# Calculate precision
matches = sum((cand_grams & ref_grams).values())
precision = matches / sum(cand_grams.values()) if cand_grams else 0
return precision # Simple 1-gram BLEU approximationMoving Beyond Tokens: Human Evaluation
Automated metrics fail to capture the subjective quality of coherence, helpfulness, and safety. Human evaluation remains the gold standard because humans can assess the 'intent' behind an output rather than just surface-level token alignment. This involves designing rubrics where human raters rank outputs on scales such as clarity, factual accuracy, and tonal appropriateness. By collecting this data, developers can fine-tune models to align with human preferences using methods like reinforcement learning from human feedback. While expensive and slow, human evaluation is necessary when the model needs to engage in open-ended creative tasks where there is no singular 'correct' token sequence. This qualitative data acts as the ultimate feedback loop, ensuring the model's statistical success translates to practical utility for the end-user in real-world scenarios where automated metrics would signal false confidence.
def record_human_feedback(prompt, response, score):
# Simple structure for gathering training data
feedback_entry = {
'prompt': prompt,
'response': response,
'rating': score, # Integer scale 1-5
'timestamp': '2023-10-27'
}
# Append to a file for supervised fine-tuning later
with open('human_evals.jsonl', 'a') as f:
import json
f.write(json.dumps(feedback_entry) + '\n')
return TrueDesigning Robust Evaluation Pipelines
A robust evaluation pipeline must combine these metrics to create a comprehensive picture of model performance. Relying on a single metric is dangerous because models can overfit to specific evaluation quirks, such as memorizing the training dataset or gaming n-gram overlap in BLEU. Your pipeline should split data into training, validation, and 'hold-out' test sets to prevent leakage. Automated metrics like perplexity should be tracked during training for early intervention if the model diverges, while BLEU and human evaluation should be performed on the final candidate outputs. By layering these techniques, you ensure that the model is mathematically stable, linguistically accurate, and practically helpful. Automation facilitates fast iteration, while human touchpoints ensure that the progress being made is aligned with the actual goals of the project rather than just inflating statistical scores on static test sets.
def run_full_pipeline(model, val_loader, test_prompt):
# Automated validation
ppl = calculate_perplexity(model, val_loader)
# Qualitative spot check
generated = model.generate(test_prompt)
print(f'Model Perplexity: {ppl.item():.2f}')
print(f'Sample Output: {generated}')
return {'perplexity': ppl.item(), 'sample': generated}Challenges in Evaluating Generation Tasks
Evaluating generative models is uniquely difficult because language is open-ended. Unlike classification tasks with fixed labels, generative models can produce many valid answers for a single prompt. If you evaluate based only on a specific ground-truth reference, you may penalize creative, correct, or contextually relevant answers that simply do not match the reference word-for-word. This phenomenon explains why low BLEU scores sometimes accompany high human satisfaction ratings. Therefore, evaluation must always include a variety of prompts and potentially 'model-based evaluation,' where a more powerful model acts as a proxy for human judgment. By continuously evolving your evaluation set to include tricky edge cases, you prevent the model from becoming brittle. The goal of evaluation is to build a reliable bridge between raw statistical performance and the complex, nuanced requirements of effective human-computer interaction.
def evaluate_diversity(model, prompts):
# Generate multiple responses for the same prompt to check variance
responses = [model.generate(p, temperature=0.7) for p in prompts]
# Calculate unique token ratio to measure linguistic diversity
unique_tokens = len(set(' '.join(responses).split()))
return unique_tokens / len(' '.join(responses).split())Key points
- Perplexity measures the predictive uncertainty of a model relative to the underlying data distribution.
- Lower perplexity scores generally indicate that the model has learned the training corpus structure effectively.
- BLEU score is an automated metric that quantifies n-gram overlap between machine and human text.
- The primary limitation of BLEU is its inability to interpret semantic meaning or synonym usage.
- Human evaluation provides the critical qualitative assessment of coherence and helpfulness required for alignment.
- Automated pipelines must use hold-out test sets to ensure that evaluation metrics remain objective.
- Generative evaluation is complex because multiple valid outputs can exist for a single prompt.
- A comprehensive evaluation strategy combines statistical metrics, n-gram matching, and human feedback loops.
Common mistakes
- Mistake: Relying solely on Perplexity as a proxy for task performance. Why it's wrong: Perplexity measures how well the model predicts the next token in a test set, but it does not account for factual accuracy or logical coherence. Fix: Use Perplexity for training iteration comparisons, but use downstream task benchmarks for final performance.
- Mistake: Using a single BLEU score to evaluate chat-based model responses. Why it's wrong: BLEU penalizes valid paraphrasing and creative wording because it relies on exact n-gram matching against a reference. Fix: Use LLM-based evaluation or human evaluation to assess semantic quality instead of exact word overlap.
- Mistake: Neglecting inter-annotator agreement in human evaluation. Why it's wrong: Human judgment is subjective; without measuring agreement (like Cohen's Kappa), results may be biased by individual rater preferences. Fix: Require multiple raters per sample and calculate agreement scores to ensure reliability.
- Mistake: Calculating BLEU scores on tokenized outputs without normalizing whitespace. Why it's wrong: Tokenization discrepancies between the model output and the reference corpus lead to artificial inflation or deflation of scores. Fix: Standardize tokenization and normalization methods across both candidate and reference texts.
- Mistake: Assuming lower Perplexity always translates to better model behavior in production. Why it's wrong: Models can become 'overfit' to specific validation datasets, resulting in low Perplexity while exhibiting poor generalization or increased toxicity. Fix: Evaluate models across diverse, unseen datasets and include adversarial testing.
Interview questions
How would you define perplexity in the context of training your own LLM?
Perplexity is a fundamental metric in our LLM course that measures how well a probability model predicts a sample. Mathematically, it is the exponentiated average negative log-likelihood of a sequence. If an LLM has low perplexity, it means the model is less 'surprised' by the text it encounters, indicating a better grasp of the training distribution. When I am training my own model, I monitor this constantly to ensure the loss is converging, as it directly reflects the model's predictive uncertainty on held-out validation data.
What is the primary purpose of the BLEU score, and why is it used in text generation tasks?
The Bilingual Evaluation Understudy (BLEU) score is used to evaluate the quality of text generated by an LLM by comparing it to one or more reference human translations or summaries. It calculates an n-gram overlap between the candidate text and the reference. For my LLM projects, I use BLEU to get a quick, automated sense of how closely my model's output aligns with expected ground truth, though I always remember it prioritizes exact word matching over semantic meaning or actual utility.
Why is human evaluation considered the 'gold standard' despite being more time-consuming than automated metrics?
While perplexity and BLEU are efficient, they fail to capture nuance, creativity, or factual accuracy. Humans can evaluate the 'vibes,' safety, and logical flow of an LLM, which are difficult to quantify with mathematical formulas. In my coursework, we emphasize human evaluation because it reveals if the model is actually helpful to a user. Automated metrics might give a high score to a syntactically correct but hallucinated response, whereas a human would correctly flag that as a failure.
Compare BLEU and Perplexity: When should you prioritize one over the other when fine-tuning your LLM?
You prioritize perplexity during the pre-training and initial fine-tuning phases because it measures the model's fundamental language modeling capability on raw text. In contrast, you use BLEU during inference-based tasks, like summarization, to see how well the model mimics specific reference structures. If I am training my LLM from scratch, I focus on minimizing perplexity to ensure it learns language patterns, but if I am building an instruction-tuned model, BLEU helps check if my output matches the expected target format.
How can you detect if your LLM is overfitting using perplexity during the training phase?
To detect overfitting, I compare the perplexity of the training set against the validation set. If the training perplexity continues to drop while the validation perplexity begins to rise, my model is starting to memorize the training data rather than generalizing patterns. I would implement an early stopping mechanism in my training script, perhaps using code like `if val_ppl > best_val_ppl: stop_training()`, to prevent the model from losing its ability to handle unseen data effectively.
In the context of creating your own LLM, how do you handle cases where BLEU scores are high but human evaluation results are poor?
This discrepancy indicates that the LLM is focusing on surface-level statistics rather than meaningful content. For instance, the model might produce a grammatically correct sentence with a high BLEU score because it recycled words from the reference, but the logical reasoning could be entirely flawed. In this scenario, I would pivot away from reliance on BLEU and implement reinforcement learning from human feedback (RLHF) or collect higher-quality, diverse, and reasoning-heavy instruction datasets to align the model’s internal representations with human-defined quality standards rather than just lexical similarity.
Check yourself
1. Which of the following scenarios best demonstrates why Perplexity might be misleading for an LLM intended for creative writing?
- A.A low Perplexity score indicates the model is successfully copying training data verbatim.
- B.High Perplexity accurately identifies models that are incapable of generating fluent text.
- C.Perplexity penalizes diverse, highly creative word choices that deviate from the most statistically probable next tokens.
- D.Perplexity measures the semantic intent of the generated story, which is irrelevant for creative prose.
Show answer
C. Perplexity penalizes diverse, highly creative word choices that deviate from the most statistically probable next tokens.
Perplexity is a probabilistic metric that favors the most likely next tokens; in creative writing, the 'best' word is often less probable. Option 0 is a specific failure mode, option 1 is false because fluency and probability differ, and option 3 is false because Perplexity cannot measure semantic intent.
2. When comparing two different model checkpoints, what is the primary limitation of using BLEU score as the sole metric?
- A.BLEU cannot handle long-form generated text.
- B.BLEU lacks a mechanism to understand semantic equivalence when different words are used to convey the same meaning.
- C.BLEU is computationally too expensive for modern evaluation pipelines.
- D.BLEU score is always 0 if the generated response is longer than the reference text.
Show answer
B. BLEU lacks a mechanism to understand semantic equivalence when different words are used to convey the same meaning.
BLEU is an n-gram overlap metric; it treats synonyms or paraphrases as errors. Option 0 is false, option 2 is false as it is computationally light, and option 3 is false as it handles length via brevity penalties.
3. In a human evaluation study, why is it critical to provide raters with a clear rubric and a 'Gold Standard' example?
- A.To ensure human raters follow the exact training process used for the LLM.
- B.To reduce subjective variance and ensure raters apply consistent criteria for quality, such as fluency or accuracy.
- C.To force humans to agree with the automated BLEU scores obtained earlier.
- D.To speed up the evaluation process so raters do not have to read the entire output.
Show answer
B. To reduce subjective variance and ensure raters apply consistent criteria for quality, such as fluency or accuracy.
Rubrics normalize human interpretation. Option 0 is irrelevant, option 2 is incorrect as humans should evaluate independently of flawed automated metrics, and option 3 would compromise the quality of the evaluation.
4. What does a significant decrease in Perplexity during the training of an LLM generally suggest about the model's status?
- A.The model is guaranteed to be more factually accurate on all real-world tasks.
- B.The model is becoming better at assigning higher probability to the ground-truth tokens in the validation set.
- C.The model has reached its optimal capacity and no further training is required.
- D.The model is definitely suffering from catastrophic forgetting.
Show answer
B. The model is becoming better at assigning higher probability to the ground-truth tokens in the validation set.
Perplexity is essentially the exponentiated cross-entropy; a decrease directly maps to better prediction of the next token. Options 0, 2, and 3 are assumptions not supported by the math of Perplexity.
5. If an LLM has a very low Perplexity but receives poor scores in a human evaluation of its helpfulness, what is the most likely issue?
- A.The model is too smart for human raters to understand.
- B.The model is optimized to predict likely sequences based on the training data but fails to align with the specific intent or instructions of the human prompt.
- C.The model has too many parameters, making it impossible to evaluate.
- D.The human evaluation rubric was incorrectly designed to favor high Perplexity.
Show answer
B. The model is optimized to predict likely sequences based on the training data but fails to align with the specific intent or instructions of the human prompt.
This highlights the 'Alignment' problem: a model can be statistically accurate (low perplexity) but unhelpful. Option 0 is an excuse, option 2 is irrelevant to evaluation, and option 3 is a theoretical contradiction.