Ten questions at a time, drawn from 175. Every answer is explained. Nothing is saved and no account is needed.
An e-commerce company wants to categorize its customers into distinct segments based on purchasing behavior without having predefined categories. Which approach is most appropriate?
Practice quiz for Machine Learning. Scores are not saved.
An e-commerce company wants to categorize its customers into distinct segments based on purchasing behavior without having predefined categories. Which approach is most appropriate?
Answer: Unsupervised clustering. Unsupervised clustering is correct because it identifies patterns in unlabeled data. Classification requires labeled categories; regression predicts continuous values; reinforcement learning is for decision sequences.
From lesson: ML Overview — Supervised, Unsupervised, Reinforcement
Why is Reinforcement Learning unique compared to the supervised and unsupervised paradigms?
Answer: It relies on the agent interacting with an environment over time. RL relies on an agent interacting with an environment to maximize rewards. Supervised/unsupervised methods typically process static data; accuracy is not guaranteed without training; RL can handle both discrete and continuous outputs.
From lesson: ML Overview — Supervised, Unsupervised, Reinforcement
In a supervised learning scenario, what is the primary purpose of holding out a test set?
Answer: To provide an unbiased evaluation of the final model's generalization. The test set provides an unbiased evaluation of generalization. Learning from more data, tuning hyperparameters, and removing noise are tasks performed on training or validation sets, not the test set.
From lesson: ML Overview — Supervised, Unsupervised, Reinforcement
Which of the following scenarios is best solved using a supervised learning regression model?
Answer: Predicting the exact price of a house based on its square footage. Predicting a house price involves outputting a continuous value, which is regression. Sentiment grouping is clustering/classification; navigation is RL; churn identification is binary classification.
From lesson: ML Overview — Supervised, Unsupervised, Reinforcement
If you are using dimensionality reduction on a dataset, which paradigm are you operating in?
Answer: Unsupervised learning. Dimensionality reduction (like PCA) is an unsupervised technique because it finds structure in data without needing target labels. Supervised, reinforcement, and semi-supervised approaches require explicit labels or reward signals.
From lesson: ML Overview — Supervised, Unsupervised, Reinforcement
What is the primary purpose of holding out a test set that is separate from the validation set?
Answer: To provide an unbiased assessment of the model's final performance on unseen data.. The test set provides a final, objective metric of generalization. Option 0 is wrong because data used for testing is removed from training; option 2 describes validation, not testing; option 3 is incorrect as testing adds complexity, not cost savings.
From lesson: Train / Validation / Test Split
If you are performing hyperparameter tuning using cross-validation, where should you perform data augmentation?
Answer: On the training folds during each iteration of cross-validation.. Augmentation is part of the training pipeline. Applying it before splitting (option 0) causes data leakage. Applying it to validation/test sets (options 1 and 3) corrupts the evaluation metric because the model shouldn't be tested on generated artificial samples.
From lesson: Train / Validation / Test Split
Why is stratified sampling preferred over simple random sampling when splitting a dataset for classification?
Answer: It guarantees that the minority class is represented in every split in the same proportion as the original.. Stratification ensures label consistency. It doesn't affect convergence speed (1), doesn't filter noise (2), and is not a methodology for problem transformation (3).
From lesson: Train / Validation / Test Split
You observe that your model performs exceptionally well on the training set but poorly on the validation set. What should you conclude regarding the split?
Answer: The split has resulted in a high variance scenario, likely due to overfitting.. This is a classic signature of overfitting. Small sample size (0) is a separate issue; random seeds are never 'invalid' (2); outlier density (3) doesn't typically cause this specific gap in performance compared to model complexity.
From lesson: Train / Validation / Test Split
When working with a dataset where each record represents a specific user, why is it crucial to use 'group-aware' splitting?
Answer: To prevent data leakage by ensuring all records for a specific user stay in the same partition.. If records from the same user appear in both training and testing, the model might 'memorize' the user instead of learning features. This is not for speed (1), nor for creating biases (0), nor for demographic segmentation (3).
From lesson: Train / Validation / Test Split
If your model performs very well on the training data but poorly on the validation data, what is the most likely scenario?
Answer: Low bias and high variance. High performance on training data implies low bias (the model learned the patterns). Poor performance on validation data suggests it failed to generalize, indicating high variance. High bias would lead to poor training performance, and low bias/low variance is the goal.
From lesson: Bias-Variance Tradeoff
What is the primary effect of increasing the regularization strength (e.g., lambda in Ridge regression) on the bias-variance trade-off?
Answer: It increases bias and decreases variance. Regularization restricts the model's flexibility. This makes it simpler (increasing bias) but less sensitive to specific training points (decreasing variance). The others are incorrect because regularization does not increase variance or decrease bias.
From lesson: Bias-Variance Tradeoff
You plot a learning curve and notice that the training error and validation error are both high and remain close to each other as the dataset grows. What should you do?
Answer: Increase the complexity of the model. High training and validation error indicates high bias (underfitting). To fix underfitting, you need a more flexible model. More data won't help with high bias, and regularization/feature reduction would only make underfitting worse.
From lesson: Bias-Variance Tradeoff
Why is it mathematically impossible to have zero bias and zero variance in a real-world machine learning application?
Answer: Because of irreducible noise in the data generating process. The total error is defined as Bias^2 + Variance + Irreducible Error. Even with a perfect model, there is always irreducible noise. Computational power, local minima, and outliers are practical challenges, not the fundamental reason for the theoretical limit.
From lesson: Bias-Variance Tradeoff
If you are using a decision tree, what happens to bias and variance as you increase the maximum depth of the tree?
Answer: Bias decreases, variance increases. As depth increases, the tree captures more intricate patterns (decreasing bias) but becomes hypersensitive to specific training examples (increasing variance). The other options contradict how model complexity affects these two components.
From lesson: Bias-Variance Tradeoff
If you are evaluating a classifier on a highly imbalanced dataset, which approach provides the most reliable performance estimate?
Answer: Stratified k-fold cross-validation. Stratified k-fold ensures that the percentage of samples for each class is preserved in every fold. Simple hold-out or random methods might lead to folds missing the minority class entirely. Leave-one-out is computationally expensive and high-variance, and Monte Carlo does not guarantee balanced class representation.
From lesson: Cross-Validation
Why must you fit your data preprocessing pipelines (like standardization) only on the training folds during cross-validation?
Answer: To prevent information leakage from the validation set into the training process. Fitting on the entire dataset incorporates information (like the mean or range) from the validation fold, which the model should not have access to. This creates a 'look-ahead' bias. Other options are incorrect because leakage is a negative trait, not a method to increase performance, and computational cost is not the primary reason.
From lesson: Cross-Validation
What is the primary trade-off when increasing the number of folds (k) in k-fold cross-validation?
Answer: Reduced bias and increased variance of the performance estimate. Higher k means each training fold is closer in size to the full dataset, reducing bias. However, the models become highly correlated because they are trained on nearly identical data, which increases the variance of the error estimate. The other options incorrectly describe the relationship between bias, variance, and complexity.
From lesson: Cross-Validation
In the context of hyperparameter tuning, where should the cross-validation process occur relative to the tuning?
Answer: Inside the hyperparameter tuning loop. Cross-validation must be nested within the tuning process to evaluate how well each set of hyperparameters generalizes. If it were outside, you would be tuning to the validation set. Using the test set for tuning invalidates it as a measure of generalization, and training without folds provides no validation signal.
From lesson: Cross-Validation
When applying k-fold cross-validation to a model with a high-degree polynomial feature transformer, when should the transformation be applied?
Answer: Inside each cross-validation fold using only training data. Feature engineering must be part of the training pipeline. If the transformation (e.g., calculating polynomial features) relies on global statistics, applying it before the split causes leakage. Applying it only to validation folds is mathematically incorrect for model training, and ignoring it would lead to an incorrect performance assessment of the pipeline.
From lesson: Cross-Validation
Why is it generally recommended to perform feature selection on the training set only during cross-validation?
Answer: To prevent information about the validation fold from leaking into the training process.. Selecting features based on the entire dataset leads to overfitting because the selection process 'sees' the validation data. The other options are incorrect because maximizing features often leads to poor generalization, and noise distribution or variance should be managed regardless of the fold.
From lesson: Feature Engineering and Selection
Which of the following scenarios describes a situation where an Interaction Feature is most beneficial?
Answer: When the effect of one input feature on the target depends on the value of another input feature.. Interaction terms represent the joint effect of features. A linear relationship (option 0) indicates redundancy, not interaction. Normal distribution (option 2) and missing values (option 3) are preprocessing concerns, not the primary driver for creating interaction features.
From lesson: Feature Engineering and Selection
When comparing L1 (Lasso) and L2 (Ridge) regularization for feature selection, what is the primary distinction?
Answer: L1 shrinks coefficients to zero, effectively performing feature selection, whereas L2 shrinks coefficients towards zero without reaching it.. L1 penalty encourages sparsity by forcing coefficients to zero. L2 shrinks them but keeps them in the model. L2 is often computationally easier due to differentiability, and both are used in linear/generalized linear contexts, not just trees.
From lesson: Feature Engineering and Selection
If a feature selection method removes a variable because it has a low variance, what is the fundamental assumption being made?
Answer: That the feature provides little discriminative information for the model.. Low variance implies the feature is nearly constant, making it uninformative for distinguishing between classes or target values. Linear correlation (0) and redundancy (3) are addressed by different methods like correlation analysis, and missing values (2) are handled by imputation.
From lesson: Feature Engineering and Selection
How does target encoding for categorical features differ from one-hot encoding in terms of model impact?
Answer: Target encoding captures the relationship between the category and the target, while one-hot encoding treats categories as unrelated binary flags.. Target encoding maps categories to their mean target value, capturing signal directly. One-hot treats them as independent indicators. Target encoding actually decreases dimensionality, and it is highly prone to overfitting, meaning option 3 is incorrect.
From lesson: Feature Engineering and Selection
Which scenario best describes when you should prioritize Precision over Recall in an imbalanced classification problem?
Answer: When you want to ensure that every identified positive case is actually positive, such as in high-stakes legal document classification.. Precision is the ratio of true positives to all predicted positives; it is critical when false alarms are costly. Option 0 describes a scenario requiring Recall. Option 2 is irrelevant to imbalanced strategies. Option 3 is factually incorrect regarding the nature of imbalanced data.
From lesson: Handling Imbalanced Data
If you are using SMOTE to handle imbalance, why is it necessary to perform it only on the training fold during cross-validation?
Answer: To avoid information leakage where knowledge from the validation set influences training parameters.. If SMOTE is applied before cross-validation, synthetic points created from validation data leak information into training. Option 0 would defeat the purpose of independent testing. Option 1 is not the primary purpose of SMOTE. Option 3 is the correct reasoning.
From lesson: Handling Imbalanced Data
What is the primary effect of using class weights in a neural network loss function for imbalanced data?
Answer: It increases the penalty for misclassifying minority class instances during the optimization process.. Class weights adjust the loss function to weigh errors on minority instances higher, forcing the gradient to prioritize those samples. Option 0 would cause extreme bias. Option 2 is the correct mechanism. Option 3 describes scaling, not class weighting.
From lesson: Handling Imbalanced Data
Why might undersampling the majority class be a better choice than oversampling the minority class in certain situations?
Answer: It prevents the model from becoming computationally overwhelmed by a massive dataset during training.. Undersampling reduces the size of the dataset, which can speed up training and reduce memory usage when the majority class is excessively large. Option 0 is impossible. Option 2 is incorrect as data selection is a trade-off. Option 3 is wrong because undersampling loses potential information.
From lesson: Handling Imbalanced Data
When evaluating a model on a highly imbalanced dataset, why is the Area Under the Precision-Recall Curve (AUPRC) generally preferred over the Area Under the ROC Curve (AUROC)?
Answer: AUPRC is more sensitive to the performance of the minority class, as it ignores True Negatives which are often overly abundant.. AUROC can be misleading in imbalanced datasets because the False Positive Rate remains low even if many minority samples are misclassified. AUPRC focuses on the performance of the positive class. Option 0 is inaccurate. Option 2 is the correct advantage. Option 3 and 4 are false.
From lesson: Handling Imbalanced Data
What is the primary effect of adding a redundant, irrelevant feature to a linear regression model on the training error?
Answer: The training error will decrease or stay the same. Adding features can only reduce or maintain the training error because the model gains more flexibility. It is not option 1 or 4 because irrelevant features don't increase error in training, and it is not option 2 because the model might accidentally fit noise in the feature.
From lesson: Linear Regression
If the residuals of a linear regression model show a clear 'U' shape when plotted against the predicted values, what does this indicate?
Answer: The relationship between the variables is likely non-linear. A 'U' shape in residuals suggests that a linear model cannot capture the curvature of the underlying relationship. It is not option 1, which relates to feature correlation, or 3 and 4, which don't cause structured patterns in residuals.
From lesson: Linear Regression
Why is it important to standardize input features before applying regularization techniques like Lasso or Ridge?
Answer: To ensure the model coefficients are on the same scale for fair penalization. Regularization adds a penalty based on the size of the coefficients; if features are on different scales, the penalty affects them unevenly. It is not 1 (though it helps, it's not the reason for regularization), and 3 and 4 are mathematically incorrect.
From lesson: Linear Regression
In the context of the Normal Equation, why might we prefer gradient descent when the number of features is extremely large?
Answer: The Normal Equation requires inverting a very large matrix, which is computationally expensive. Matrix inversion is O(n^3), making it infeasible for very high-dimensional data. Gradient descent is O(kn^2), making it more scalable. The other options are incorrect as the Normal Equation is an exact analytical solution and not inherently less accurate.
From lesson: Linear Regression
Which of the following best describes the bias-variance tradeoff in linear regression?
Answer: High variance models fit the training data very well but fail to generalize to new data. High variance models overfit training data. Option 1 describes high variance, not bias. Option 2 is wrong because increasing complexity usually increases variance. Option 4 is wrong because simple linear regression is usually a high-bias, low-variance model.
From lesson: Linear Regression
What is the primary purpose of the sigmoid function in logistic regression?
Answer: To transform the linear combination of inputs into a probability range between 0 and 1. The sigmoid function maps any real-valued number into the (0, 1) interval, which represents the probability of a binary outcome. Option 1 is incorrect as normalization is a preprocessing step, not the model's function. Option 2 is wrong because the output must be a probability, not an integer. Option 4 is false as optimization is still required.
From lesson: Logistic Regression
Why is the Mean Squared Error (MSE) generally not used as a loss function for logistic regression?
Answer: It leads to a non-convex optimization surface which makes finding the global minimum difficult. With the sigmoid function, MSE leads to a non-convex cost function with many local minima, making gradient descent unreliable. Log Loss is convex, ensuring a unique global minimum. The other options are incorrect as they do not address the mathematical properties of the cost surface.
From lesson: Logistic Regression
When interpreting the coefficients of a logistic regression model, what does a positive coefficient indicate?
Answer: The probability of the positive class increases as the feature value increases. In logistic regression, a positive coefficient indicates that the log-odds of the positive class increase as the feature value increases, thus increasing the probability. Option 0 is wrong because the relationship is probabilistic, not linear. Option 2 is false by definition of a coefficient's purpose. Option 3 is unrelated to the sign of the coefficient.
From lesson: Logistic Regression
How does adding L2 regularization (Ridge) affect the weights of a logistic regression model?
Answer: It penalizes large weights to keep them small and reduce model complexity. L2 regularization adds a penalty proportional to the square of the weights, which prevents them from becoming too large and helps generalize better. Option 0 describes L1 (Lasso) regularization. Option 1 is incorrect as weights can be positive. Option 3 is wrong because regularization does not directly affect the learning rate.
From lesson: Logistic Regression
In a binary classification task, if the logistic regression output for a data point is 0.8, what does this signify?
Answer: The model is 80% confident the observation belongs to the positive class. The output of logistic regression is the predicted probability of the positive class. 0.8 means the model predicts an 80% chance of the positive class. Option 1 is imprecise because a classification decision usually requires a specific threshold (e.g., 0.5). Option 2 is a misinterpretation of probability. Option 3 is logically unrelated.
From lesson: Logistic Regression
If a decision tree node is perfectly pure, what is the value of the Gini Impurity?
Answer: 0. A pure node contains only one class, resulting in a Gini Impurity of 0. 0.5 represents maximum impurity in a binary classification, 1 is impossible for Gini, and Infinity is not a defined value for this metric.
From lesson: Decision Trees
What is the primary effect of increasing the 'min_samples_split' hyperparameter?
Answer: Increased regularization and reduced overfitting. Increasing this parameter forces the tree to stop splitting if a node has fewer samples, which reduces complexity (regularization) and prevents overfitting. It does not necessarily decrease training time, and it specifically aims to avoid capturing noise.
From lesson: Decision Trees
In a regression tree, what value is typically assigned to a leaf node during prediction?
Answer: The mean of target values in the leaf. Regression trees predict the mean of the target variable for all training instances that fall into the leaf. Median is sometimes used for robustness, but mean is the standard for minimizing squared error. Max is never used, and mode is for classification.
From lesson: Decision Trees
Why do decision trees perform well even when data features are on completely different scales?
Answer: Because splits are based on rank order and thresholding of individual features. Decision trees split based on whether a feature value is greater than a threshold; they do not calculate distances (unlike KNN) or apply weights to features (unlike linear models), making scale irrelevant. They do not perform automatic normalization or gradient descent.
From lesson: Decision Trees
Which of the following scenarios describes the 'High Variance' problem in a single decision tree?
Answer: The model performs well on training data but poorly on test data. High variance means the model is overly sensitive to training data fluctuations (overfitting), performing well on training sets but poorly on new data. The other options describe underfitting or high bias scenarios.
From lesson: Decision Trees
Why does a Random Forest typically exhibit lower variance than a single decision tree?
Answer: It averages the predictions of multiple deep trees trained on different bootstrap samples.. Averaging decorrelated trees reduces variance without significantly increasing bias. Option 0 describes Gradient Boosting, option 2 is incorrect because feature sets overlap, and option 3 describes a single global model, not an ensemble.
From lesson: Random Forests
What is the primary effect of the 'max_features' hyperparameter in a Random Forest?
Answer: It influences the correlation between trees by limiting the feature set per split.. By limiting the features at each split, the trees become less correlated, which is the core strength of the ensemble. Option 0 refers to 'n_estimators', option 1 refers to 'max_depth', and option 3 is not a standard hyperparameter.
From lesson: Random Forests
What does the 'Out-of-Bag' (OOB) error represent in a Random Forest?
Answer: The error rate on samples that were not included in the bootstrap sample of a specific tree.. OOB allows for model validation during training without a separate validation set. Option 0 is a standard test set, while 2 and 3 do not define OOB.
From lesson: Random Forests
When comparing a Random Forest to a single Decision Tree, which statement is most accurate regarding bias and variance?
Answer: Variance is reduced, while bias remains relatively similar.. Bagging (the foundation of Random Forest) primarily targets variance reduction. Bias usually remains similar to that of a single unpruned decision tree. Option 0 is wrong because bias isn't the primary target, and 3 is wrong because the bias does not increase.
From lesson: Random Forests
How does Random Forest handle feature importance, and what is its main limitation?
Answer: It calculates importance based on the reduction in impurity, but can be biased toward high-cardinality features.. Impurity-based importance is standard but favors variables with many categories. Option 1 is incorrect because permutation importance is separate, 2 is incorrect because forests aren't linear models, and 3 is incorrect because depth does not determine importance.
From lesson: Random Forests
When transitioning from XGBoost to LightGBM, why might a model configured with the exact same depth parameters overfit significantly more in LightGBM?
Answer: LightGBM uses leaf-wise growth, which allows for more complex, asymmetric trees at the same maximum depth.. LightGBM's leaf-wise growth strategy focuses on splitting the leaf that reduces loss the most, leading to deeper, more complex trees compared to XGBoost's level-wise approach. Option 1 is wrong because LightGBM does respect depth; Option 3 is incorrect as both have tunable regularization; Option 4 is irrelevant to complexity.
From lesson: Gradient Boosting — XGBoost, LightGBM
A model is showing low training error but high validation error. Which parameter adjustment is most likely to improve generalization by directly reducing model complexity?
Answer: Decreasing the 'colsample_bytree' or 'feature_fraction' parameters.. Reducing 'colsample_bytree' or 'feature_fraction' introduces randomness and limits the model's reliance on specific, potentially noisy features. Increasing the learning rate (Option 1) usually makes convergence harder; increasing bins (Option 2) increases complexity; increasing estimators (Option 4) increases overfitting.
From lesson: Gradient Boosting — XGBoost, LightGBM
In the context of Gradient Boosting, what is the mathematical function of the 'learning rate' parameter?
Answer: It acts as a shrinkage factor for the contribution of each new weak learner to the ensemble.. The learning rate (eta) scales the contribution of each subsequent tree, forcing the model to take smaller steps toward the objective function minimum. Options 2, 3, and 4 refer to initialization, splitting criteria, and tree constraints respectively, not the shrinkage mechanism.
From lesson: Gradient Boosting — XGBoost, LightGBM
Why is the Histogram-based approach used in modern Gradient Boosting frameworks like LightGBM and XGBoost preferred over exact greedy split finding for large datasets?
Answer: It significantly reduces the computational cost by binning continuous features into discrete buckets.. Binning continuous features reduces the number of candidate split points from O(N) to O(bins), drastically speeding up training on large data. Option 1 is incorrect as binning can lose precision; Option 3 is false; Option 4 is not the primary purpose of histogram binning.
From lesson: Gradient Boosting — XGBoost, LightGBM
Which of the following scenarios best justifies the use of 'scale_pos_weight' (XGBoost) or 'is_unbalance' (LightGBM)?
Answer: When the dataset has a significant imbalance between the positive and negative classes.. These parameters compensate for class imbalance by assigning higher weight to the minority class in the loss calculation. Option 1 is for regression; Option 3 is handled by default sparsity handling; Option 4 is irrelevant since gradient boosting is scale-invariant.
From lesson: Gradient Boosting — XGBoost, LightGBM
If you are training an SVM on a dataset where the number of features (n) is much larger than the number of training samples (m), which kernel should you choose?
Answer: A linear kernel. With a linear kernel, you avoid overfitting in high-dimensional spaces where the features are already sufficient to linearly separate the classes. RBF or complex kernels would likely overfit the small sample size (m).
From lesson: Support Vector Machines (SVM)
What is the primary effect of increasing the 'C' hyperparameter in a soft-margin SVM?
Answer: The model attempts to classify all training points correctly, narrowing the margin.. A larger C reduces the penalty for margin violations but increases the penalty for misclassifications, forcing the boundary to fit the training data tightly. The other options describe effects of lower C or are incorrect SVM theory.
From lesson: Support Vector Machines (SVM)
Why is the 'Kernel Trick' computationally efficient?
Answer: It allows the model to calculate distances in high-dimensional space without explicitly transforming the data.. The kernel trick computes the inner product in the feature space directly from the original space, avoiding the expensive computation of the high-dimensional coordinates. It does not reduce samples, change the optimization problem type, or perform feature selection.
From lesson: Support Vector Machines (SVM)
How does an SVM define the 'optimal' hyperplane?
Answer: The hyperplane that maximizes the distance (margin) between the nearest points of each class.. SVMs define the boundary by maximizing the margin to ensure robustness. Accuracy minimization (option 1) is common in other models but not the objective of SVM, and minimizing squared error (option 2) is characteristic of regression, not SVM classification.
From lesson: Support Vector Machines (SVM)
If a dataset has highly overlapping classes, which SVM approach is most appropriate?
Answer: Using a Soft Margin SVM with a low C value.. A Soft Margin SVM with a low C allows the model to ignore noisy data points (overlap), which prevents the model from trying to create a complex, overfitted boundary. Hard margin would fail if the data is not linearly separable, and the other options do not address the optimization goal.
From lesson: Support Vector Machines (SVM)
How does increasing the value of K affect the decision boundary of a KNN classifier?
Answer: It makes the decision boundary smoother and more generalized. Increasing K considers more neighbors, which averages out the noise and leads to a smoother decision boundary, reducing variance. A small K makes the boundary jagged (overfitting), not smoother. K has a significant effect, and it does not force linearity.
From lesson: K-Nearest Neighbors (KNN)
Why is it critical to standardize features before applying KNN?
Answer: To prevent features with large scales from dominating the distance calculation. KNN relies on distance metrics; if one feature ranges from 0-1000 and another from 0-1, the first feature will dictate the distance calculation. Standardizing puts them on the same scale. This does not change complexity, fix categorical issues, or handle missing data.
From lesson: K-Nearest Neighbors (KNN)
Which of the following describes the behavior of KNN when the training set size increases significantly?
Answer: The prediction time increases significantly. KNN is a lazy learner; at prediction time, it must compare the query point to all training points, so prediction time grows linearly with the number of training samples. Training time is zero, not faster. Larger datasets usually improve accuracy, and memory requirements remain constant or grow, they do not decrease.
From lesson: K-Nearest Neighbors (KNN)
What happens when K is set to the total number of training samples?
Answer: The model will always predict the majority class of the entire dataset. If K equals the number of training samples, the model considers every point in the training set for every prediction, effectively predicting the majority class of the entire dataset. It is not perfect, it does not only consider the nearest neighbor (which is K=1), and it is not a linear regression.
From lesson: K-Nearest Neighbors (KNN)
In the context of KNN, what is a primary drawback of using a very high-dimensional feature space?
Answer: Distance metrics become meaningless because all points appear equidistant. In high dimensions, the distance between any two points tends to converge to a similar value, making 'nearest' neighbors indistinguishable from 'farthest' ones. This is the 'Curse of Dimensionality'. It does not inherently prevent categorical usage, simplify the model (it actually causes overfitting), or make mean calculation impossible.
From lesson: K-Nearest Neighbors (KNN)
What is the primary consequence of the 'Naive' independence assumption on the model's decision boundary?
Answer: It allows the model to estimate high-dimensional joint distributions using one-dimensional marginals. The independence assumption simplifies the joint probability calculation to a product of marginals, making it computationally feasible to handle many features. The other options are incorrect because Naive Bayes can form non-linear boundaries, considers priors, and is highly effective for binary classification.
From lesson: Naive Bayes
If you are applying Naive Bayes to a text classification task where features are word counts, which variant is most appropriate?
Answer: Multinomial Naive Bayes. Multinomial Naive Bayes is designed for count-based data (frequencies of occurrences). Gaussian is for continuous data, Bernoulli is for boolean/binary occurrence (presence/absence), and Complement is a specific adaptation for imbalanced datasets, not the standard for word counts.
From lesson: Naive Bayes
Why does the 'Zero Frequency' problem occur in the Multinomial model and how is it resolved?
Answer: A feature value never appears with a class; use Laplace smoothing. If a word never appears in a specific category, the probability is zero, which zero-out the entire posterior. Laplace smoothing adds a small count (usually 1) to all feature frequencies to prevent zero probabilities. Increasing data or oversampling doesn't guarantee the removal of zeros.
From lesson: Naive Bayes
When comparing Naive Bayes to Logistic Regression, which statement is most accurate regarding their learning approach?
Answer: Naive Bayes is a generative model, while Logistic Regression is discriminative. Naive Bayes is a generative model because it models the joint probability P(X, Y), while Logistic Regression is a discriminative model that models the conditional probability P(Y|X) directly. The other options reverse these definitions.
From lesson: Naive Bayes
Why are log-probabilities used instead of raw probabilities during the classification stage?
Answer: To avoid numerical underflow caused by multiplying many small fractions. Multiplying many probabilities (values < 1) causes numbers to become so small they exceed the precision limit of computer memory (underflow). Using log(p1 * p2) = log(p1) + log(p2) turns multiplication into addition and keeps the values within a manageable numerical range. The other options do not describe the primary computational purpose of log-transforming probabilities.
From lesson: Naive Bayes
If you are running K-means on a dataset where one feature represents 'Annual Income' (in thousands) and another represents 'Age' (in years), what is the most critical preprocessing step to ensure fairness in clustering?
Answer: Standardizing the features to have a mean of 0 and a variance of 1. Standardizing is correct because features with larger scales would disproportionately influence the distance calculations. Log transforms help with skewness, not scale; one-hot encoding is for categorical data; outlier removal is good but does not solve the scale variance issue.
From lesson: K-Means Clustering
Which of the following scenarios best explains why K-means might fail to produce useful clusters?
Answer: The clusters have significantly different sizes and irregular shapes. K-means assumes spherical clusters of similar density and size. Irregular shapes cause the centroid to shift incorrectly. High dimensionality affects computation but not clustering utility directly; overlapping clusters are a property of the data, and large datasets are usually where K-means excels.
From lesson: K-Means Clustering
What is the role of the 'inertia' metric in the context of K-means?
Answer: It represents the sum of squared distances of samples to their closest cluster center. Inertia (within-cluster sum-of-squares) measures how compact the clusters are. Centroid distance is not the primary objective; probability is used in Soft Clustering; iteration count is a performance metric, not a cluster quality metric.
From lesson: K-Means Clustering
How does the selection of initial centroid locations affect the K-means algorithm?
Answer: It can cause the algorithm to converge to different local optima. K-means is a local search algorithm. Poor initialization can trap it in suboptimal configurations. It does not guarantee the global optimum, it certainly affects assignments, and it impacts both speed and the final result.
From lesson: K-Means Clustering
When using the Elbow Method to choose K, what are you looking for in the plot of inertia vs. number of clusters?
Answer: The point where the rate of decrease in inertia significantly levels off. The 'elbow' represents a point of diminishing returns where adding more clusters provides little additional benefit in reducing variance. Inertia never hits zero unless K equals the number of samples; the high point is simply K=1; the final option is a mathematical impossibility for most datasets.
From lesson: K-Means Clustering
Which of the following scenarios describes the 'chaining effect' often observed in hierarchical clustering?
Answer: The single linkage method merges clusters based on the smallest distance between any two points, causing elongated, thin clusters to merge easily.. The chaining effect is a known artifact of single linkage clustering where distant points are connected via a chain of intermediate points. Option 0 describes centroid/Ward's bias, Option 2 is a generic limitation, and Option 3 is a data visualization issue, not a clustering property.
From lesson: Hierarchical Clustering
How does the Ward linkage method differ from the Complete linkage method?
Answer: Ward linkage focuses on minimizing the increase in total within-cluster variance, while Complete linkage focuses on the maximum distance between clusters.. Ward's method is designed to minimize variance, which generally leads to compact, equally sized clusters. Complete linkage looks at the furthest pair of points. The other options misrepresent the mathematical objectives of the linkages.
From lesson: Hierarchical Clustering
If you are using hierarchical clustering to identify nested groups, what is the most significant advantage compared to K-Means?
Answer: It does not require pre-specifying the number of clusters to generate the full tree structure.. Hierarchical clustering provides a dendrogram that reveals cluster relationships at multiple scales without needing an initial 'k'. K-Means requires 'k' in advance, is faster but not guaranteed global, and neither handles categorical data natively without encoding.
From lesson: Hierarchical Clustering
Why might a practitioner choose to use Manhattan distance over Euclidean distance in hierarchical clustering?
Answer: Manhattan distance is more robust to high-dimensional datasets and helps mitigate the curse of dimensionality by avoiding squared differences.. In high dimensions, squared Euclidean distance (used in many algorithms) often loses contrast. Manhattan distance often provides better separation. It is not necessarily faster, doesn't force circularity, and Ward's can work with others.
From lesson: Hierarchical Clustering
What happens if you cut a dendrogram at different heights?
Answer: The clusters change from a more granular, fine-grained grouping to a coarser, more consolidated grouping.. Cutting at a higher level merges clusters, reducing the count (coarser). Cutting lower keeps them separated (finer). The linkage and distance matrix remain static once calculated.
From lesson: Hierarchical Clustering
If you increase the 'epsilon' parameter in DBSCAN while keeping 'minPts' constant, what is the most likely effect on the clustering result?
Answer: The number of clusters will decrease and noise points will likely be absorbed into clusters.. Increasing epsilon expands the neighborhood search radius. More points satisfy the density criteria, leading to larger clusters and fewer noise points. The other options are incorrect because they describe a decrease in sensitivity rather than the expansion of boundaries.
From lesson: DBSCAN
What is the primary reason DBSCAN is considered robust to outliers compared to K-Means?
Answer: DBSCAN does not force every data point into a cluster, allowing low-density points to be labeled as noise.. DBSCAN explicitly labels points that do not meet density requirements as noise. K-Means forces every point into a cluster based on proximity to a centroid. The other options are incorrect because DBSCAN is non-parametric and does not perform automated scaling or centroid-based clustering.
From lesson: DBSCAN
In the context of DBSCAN, what defines a 'border point'?
Answer: A point that is within the epsilon radius of a core point but has fewer than 'minPts' neighbors.. A border point is reachable from a core point but does not have enough neighbors itself to be a core point. The first option describes a core point, the second is unrelated to DBSCAN, and the fourth describes a centroid, which DBSCAN does not use.
From lesson: DBSCAN
Why might DBSCAN struggle with datasets containing clusters of significantly different densities?
Answer: Because it relies on a single epsilon value that cannot distinguish between low-density and high-density clusters simultaneously.. DBSCAN uses a fixed epsilon for the entire dataset. If epsilon is set for dense clusters, sparse clusters appear as noise; if set for sparse, dense clusters merge. The other options are wrong because DBSCAN does not require specifying the number of clusters, does not assume spherical clusters, and has polynomial time complexity.
From lesson: DBSCAN
How does increasing the 'minPts' parameter generally affect the output of DBSCAN?
Answer: It makes the algorithm more strict, requiring higher local density to form a cluster.. Increasing minPts raises the barrier for a point to be considered a 'core point,' making the algorithm more conservative. Option 1 is wrong because it makes it more strict. Option 3 is wrong because higher density checks do not inherently increase speed, and Option 4 is wrong as it results in more, not fewer, noise points.
From lesson: DBSCAN
What is the primary geometric effect of PCA on a dataset?
Answer: It rotates the coordinate system to align with the directions of maximum variance.. PCA rotates the axes to align with the eigenvectors of the covariance matrix. The first option describes non-linear methods; the third describes outlier detection; the fourth is incorrect because PCA works on numerical, not categorical, features.
From lesson: Dimensionality Reduction — PCA
If you have a dataset with highly correlated features, how does PCA handle this?
Answer: It identifies these correlations and merges them into a smaller set of uncorrelated components.. PCA orthogonalizes correlated features by finding principal components that have zero covariance. The first option is false because PCA is linear. The second is false because PCA doesn't increase variance. The fourth is false because it ignores the collective information.
From lesson: Dimensionality Reduction — PCA
When should you prefer scaling your features before performing PCA?
Answer: When the features have different scales and ranges, such as age and annual income.. Scaling is crucial when features have different units because PCA is sensitive to variance magnitude. If you don't scale, the feature with the largest numerical range dominates the PCA. The other options suggest scaling is unnecessary or harmful, which is incorrect.
From lesson: Dimensionality Reduction — PCA
How do you decide how many principal components to retain in a model?
Answer: Choose the number of components that explain a sufficient cumulative percentage of the total variance.. The cumulative explained variance ratio is the standard metric for dimensionality reduction. Retaining n-1 is inefficient. Selecting the smallest eigenvalues discards the most important information. Missing values have no relation to component selection criteria.
From lesson: Dimensionality Reduction — PCA
Why might a Machine Learning practitioner choose to use PCA before training a linear regression model?
Answer: To mitigate multicollinearity issues among independent features.. Multicollinearity makes linear regression coefficients unstable; PCA removes this by creating orthogonal features. Cross-validation is still needed. PCA makes the model simpler, not more complex, and cannot induce non-linearity.
From lesson: Dimensionality Reduction — PCA
Which statement best describes the fundamental difference in how t-SNE and UMAP handle global structure?
Answer: UMAP creates a fuzzy topological representation of the data, while t-SNE emphasizes only pair-wise local distances.. UMAP is built on a solid theoretical framework of manifold learning and fuzzy simplicial sets that better preserve global structure than t-SNE. The first option is partially correct regarding the distribution, but misses the core point; the second is false (both are stochastic); the fourth is incorrect as UMAP is generally faster than standard t-SNE.
From lesson: t-SNE and UMAP
If you increase the 'perplexity' hyperparameter in t-SNE, what is the expected outcome?
Answer: The algorithm will balance local and global structure by considering a larger set of neighboring points.. Perplexity is a measure of the effective number of neighbors; increasing it allows each point to see more of its surroundings, which forces the model to respect broader global structures. The other options are incorrect because perplexity does not influence linear constraints or computational speed linearly.
From lesson: t-SNE and UMAP
What is the primary reason why t-SNE/UMAP visualizations are often criticized in a rigorous scientific setting?
Answer: They are sensitive to the random seed and hyperparameter choices, which can create 'phantom' structures.. Because these methods are stochastic, running them with different seeds can lead to significantly different visual clusterings, making them prone to 'cherry-picking' results. Other options are false: they handle high dimensions well, they are unsupervised, and they don't explicitly require labels.
From lesson: t-SNE and UMAP
Why is it generally discouraged to use t-SNE or UMAP as a feature extraction step for training a supervised regression model?
Answer: These methods create non-linear transformations that are not easily interpretable or generalizable to test data.. Non-linear manifold learning transforms the input space in ways that are specific to the training set; applying these to new test data is not straightforward, and the resulting coordinates are often complex combinations of features. The other options are either false (they produce continuous output) or misunderstanding the purpose of variance.
From lesson: t-SNE and UMAP
Which of the following scenarios is ideal for choosing UMAP over t-SNE?
Answer: When you want to balance computational efficiency with preservation of global data structure.. UMAP is generally faster and better at preserving global structure than t-SNE, making it ideal for larger datasets where global relationships matter. Small datasets don't strictly prefer UMAP, it is not strictly deterministic, and PCA is the correct choice for principal components.
From lesson: t-SNE and UMAP
A medical model detects rare diseases. Which metric is most critical if missing a sick patient has a severe negative health outcome?
Answer: Recall. Recall is essential because it minimizes False Negatives, ensuring sick patients are identified. Accuracy fails on imbalanced data, Precision focuses on reducing false alarms, and Specificity focuses on negative classes.
From lesson: Classification Metrics — Accuracy, Precision, Recall, F1
In a spam filter where incorrectly blocking a legitimate email is considered a disaster, which metric should you prioritize?
Answer: Precision. Precision is key here to reduce False Positives (legitimate emails marked as spam). Recall would prioritize catching all spam even at the risk of blocking good mail, F1-score balances both, and Accuracy ignores the cost of specific error types.
From lesson: Classification Metrics — Accuracy, Precision, Recall, F1
What happens to the performance metrics if you increase the classification threshold for a model?
Answer: Precision increases and Recall decreases. Increasing the threshold makes the model more selective; it only predicts positive if very confident, which boosts Precision but makes it more likely to miss positives, thus decreasing Recall.
From lesson: Classification Metrics — Accuracy, Precision, Recall, F1
Why is the F1-score defined as the harmonic mean rather than the arithmetic mean of Precision and Recall?
Answer: It penalizes models that have very low scores in either precision or recall. The harmonic mean is sensitive to small values; if either Precision or Recall is near zero, the F1-score drops drastically. The arithmetic mean would inaccurately inflate the score if one metric is high and the other is low.
From lesson: Classification Metrics — Accuracy, Precision, Recall, F1
If your model has 100% Accuracy but performs poorly on the minority class, what does this indicate?
Answer: The dataset is likely heavily imbalanced and the model predicts only the majority class. High accuracy on imbalanced data usually means the model is just predicting the most frequent class. If the model ignored the minority class, it would never achieve 100% precision or recall on that class, making the F1-score low.
From lesson: Classification Metrics — Accuracy, Precision, Recall, F1
What does the x-axis of an ROC curve represent, and how does it react as you decrease the classification threshold?
Answer: False Positive Rate; it increases.. The x-axis is the False Positive Rate. Decreasing the threshold classifies more instances as positive, increasing both True Positives and False Positives. Options 1 and 3 are incorrect because they name the wrong metrics. Option 4 is wrong because the FPR increases, not decreases, as the threshold is lowered.
From lesson: ROC Curve and AUC-ROC
Which scenario would result in an AUC-ROC score of less than 0.5?
Answer: A model that has a strong negative correlation with the true labels.. An AUC below 0.5 indicates the model is performing worse than random guessing, essentially doing the opposite of what is required; flipping the output would make it a good model. Option 1 results in 0.5 exactly. Options 3 and 4 do not guarantee an AUC below 0.5.
From lesson: ROC Curve and AUC-ROC
Why is the ROC curve considered 'invariant' to class distribution changes in the test set?
Answer: Because it uses True Positive Rate and False Positive Rate, which are calculated using row-wise normalization of the confusion matrix.. FPR and TPR are calculated from the confusion matrix columns (TP/P and FP/N), making them independent of the total count of positive or negative instances. Options 1 and 4 are wrong definitions, and option 3 is incorrect as ROC is explicitly used when Accuracy is misleading.
From lesson: ROC Curve and AUC-ROC
If two different models have the same AUC-ROC score, what can you conclude about their performance?
Answer: They have the same average ranking ability across all possible thresholds, but may differ in specific operating points.. AUC-ROC aggregates performance; two models can have crossing ROC curves and identical areas while performing differently at specific thresholds. Options 1 and 2 are incorrect because AUC is a summary statistic, not a point-wise identity. Option 4 is an assumption that is not necessarily true.
From lesson: ROC Curve and AUC-ROC
In a clinical setting where you want to identify as many patients with a disease as possible (high sensitivity), which part of the ROC curve should you look at?
Answer: The area near the top-left corner of the plot.. Sensitivity is the True Positive Rate (y-axis). To maximize sensitivity, you choose a threshold that puts you at the top of the curve. To keep the False Positive Rate (x-axis) low, you stay as far left as possible. The other options either represent poor recall or poor precision/high false alarm rates.
From lesson: ROC Curve and AUC-ROC
If a model's RMSE is significantly higher than its MAE, what does this indicate about the distribution of prediction errors?
Answer: The model has a few large errors that are being heavily penalized. RMSE squares the errors, meaning larger residuals contribute exponentially more to the final score than smaller ones. Option 1 is incorrect because magnitude does not imply direction. Option 3 is incorrect as this would make both metrics near zero. Option 4 is incorrect because large differences between RMSE and MAE suggest the presence of outliers.
From lesson: Regression Metrics — MAE, RMSE, R²
Why is the R² score often criticized for use in multiple regression models?
Answer: It tends to increase even when irrelevant features are added to the model. R² does not account for the number of predictors, so adding noise can artificially inflate the score. Option 1 is a limitation but not the primary criticism for model evaluation. Option 2 is false as R² depends on variance. Option 4 is incorrect as R² equals 0 in that scenario, which is well-defined.
From lesson: Regression Metrics — MAE, RMSE, R²
A model achieves an R² of 0.85 on a test set. How should this be interpreted?
Answer: The model explains 85% of the variance in the target variable. R² represents the proportion of variance explained. Option 1 refers to absolute error, not variance. Option 3 is incorrect because R² is relative to the mean, not random guessing. Option 4 is incorrect because R² is rarely exactly 1 in real-world scenarios.
From lesson: Regression Metrics — MAE, RMSE, R²
When is it most appropriate to use MAE instead of RMSE?
Answer: When the dataset contains extreme outliers that you want to ignore. MAE treats all errors linearly, reducing the impact of outliers compared to the squared penalty of RMSE. Option 1 is wrong as MAE is for regression. Option 2 describes RMSE. Option 4 is incorrect as residual distribution is an assumption of the model, not a requirement for the metric.
From lesson: Regression Metrics — MAE, RMSE, R²
What happens to MAE if you shift all target values by a constant C?
Answer: MAE remains exactly the same. MAE measures the average absolute difference between predicted and actual values. If both are shifted by C, the difference (Y - Y_hat) remains unchanged. Options 1, 2, and 4 are mathematically incorrect based on the definition of subtraction.
From lesson: Regression Metrics — MAE, RMSE, R²
In a model predicting a rare disease (1% prevalence), your model predicts 'Negative' for every single patient. What does this reveal about your metrics?
Answer: The accuracy is 99%, but the recall for the positive class is 0%.. With 99% of patients healthy, predicting 'Negative' always results in 99% accuracy. However, because it never identifies a positive case, the recall is 0%. Option 1 is wrong because accuracy is high. Option 3 is wrong because precision for the positive class is undefined (no positive predictions). Option 4 is wrong because F1-score is 0 when recall is 0.
From lesson: Confusion Matrix
If you increase the classification threshold for a positive class, what is the expected impact on the confusion matrix?
Answer: False Positives will decrease, and True Positives will decrease.. Raising the threshold makes the model more 'conservative', leading to fewer positive predictions. Thus, False Positives drop, but fewer actual positives are captured, causing True Positives to also drop. Options 1, 2, and 4 contradict the mathematical relationship between threshold, FP, and TP.
From lesson: Confusion Matrix
When comparing two models on a balanced dataset, Model A has higher Precision, while Model B has higher Recall. How should you choose?
Answer: Calculate the F1-score to find a balance between the two.. Precision and Recall are often in tension. The F1-score provides the harmonic mean of both, allowing a fair comparison. Options 1 and 2 are context-dependent. Option 4 is irrelevant as high FP counts usually indicate poor performance.
From lesson: Confusion Matrix
In a 3-class confusion matrix, what does a value in row 2, column 3 represent?
Answer: The number of instances that were actually class 2 but were predicted as class 3.. By convention, rows represent actual labels and columns represent predicted labels. Row 2, column 3 indicates actual class 2 misclassified as predicted class 3. Option 1 is correct; others misidentify the row-column relationship.
From lesson: Confusion Matrix
What does a False Negative represent in the context of a spam filter?
Answer: A spam email marked as legitimate.. A False Negative means the model failed to detect the positive class (spam). Option 0 is a False Positive. Options 2 and 3 are correct classifications (True Negatives and True Positives respectively).
From lesson: Confusion Matrix
What is the primary benefit of using a Pipeline object during cross-validation?
Answer: It ensures that preprocessing parameters are learned only from the training fold and applied to the validation fold. The correct answer is the second option because it prevents data leakage during cross-validation. The other options are incorrect because pipelines do not inherently speed up training, do not optimize hyperparameters (that is GridSearch), and do not fix missing values without explicit transformers.
From lesson: Scikit-learn Pipeline
If you have a pipeline with a scaler and a classifier, what happens when you call pipeline.predict(X_test)?
Answer: It transforms X_test using the parameters learned during the fit process, then predicts. The second option is correct because the pipeline stores the state (e.g., mean and standard deviation) from the training set and applies the exact same transformation to the test set. The others are wrong because refitting on the test set causes leakage, it doesn't require a fit on the test set, and it definitely applies all steps.
From lesson: Scikit-learn Pipeline
Why is ColumnTransformer often used in conjunction with a Pipeline?
Answer: To apply different preprocessing techniques to different subsets of columns simultaneously. ColumnTransformer is designed to apply different transformers to different columns (e.g., scaling numeric data while one-hot encoding categorical data). The other options are wrong because ColumnTransformer is not for model selection, memory management, or visualization.
From lesson: Scikit-learn Pipeline
When hyperparameter tuning a pipeline using GridSearchCV, how do you specify a parameter for a step named 'scaler'?
Answer: scaler__with_mean. The double underscore syntax is the standard convention to access parameters of a step inside a pipeline. The other options use incorrect separators or invalid formatting for the GridSearchCV parameter grid.
From lesson: Scikit-learn Pipeline
If your pipeline includes a feature selection step and a classifier, what is the best practice for tuning the number of features?
Answer: Include the number of features as a parameter in the grid search. Including the feature selection parameter in the grid search allows the model to find the optimal number of features for the specific classifier used. Hardcoding or manual selection prevents the pipeline from finding the best global configuration.
From lesson: Scikit-learn Pipeline
When comparing GridSearch and RandomSearch, which statement best characterizes their behavior?
Answer: RandomSearch is more efficient when some hyperparameters are significantly more important than others.. RandomSearch is effective because it samples from the parameter space, which is better when a few parameters drive model performance. GridSearch is computationally expensive and wastes time on unimportant parameters, and neither guarantees a global optimum.
From lesson: Hyperparameter Tuning — GridSearch, RandomSearch, Optuna
How does Bayesian optimization (used in Optuna) differ from traditional grid or random approaches?
Answer: It treats hyperparameter tuning as a black-box function and uses past evaluations to choose the next set of parameters.. Bayesian optimization builds a probabilistic model of the objective function, using it to select promising parameters. The other options describe random selection, incorrect gradient usage, or brute-force search.
From lesson: Hyperparameter Tuning — GridSearch, RandomSearch, Optuna
If your model performance improves during cross-validation but degrades significantly on a hold-out test set, what is the most likely culprit?
Answer: The hyperparameters were tuned specifically to minimize the error on the validation folds, leading to validation-set overfitting.. Overfitting the validation set occurs when the tuning process 'picks up' on the noise in the validation folds. The other options describe underfitting or search space limitations, not generalizability issues.
From lesson: Hyperparameter Tuning — GridSearch, RandomSearch, Optuna
What is the primary benefit of using pruning in a framework like Optuna?
Answer: It stops unpromising trials early to save computational resources for more hopeful configurations.. Pruning terminates trials that show poor performance early on, allowing the search to focus on regions of the space that are likely to yield better models. It does not directly affect feature selection or model size.
From lesson: Hyperparameter Tuning — GridSearch, RandomSearch, Optuna
Why might you prefer a log-uniform distribution over a uniform distribution when searching for a hyperparameter like learning rate?
Answer: The impact of a learning rate change is often proportional to its magnitude; log-uniform covers multiple orders of magnitude effectively.. Learning rate optimization often requires searching across different scales (e.g., 0.001 vs 0.01 vs 0.1). A uniform distribution covers absolute differences, while a log-uniform covers relative differences, which are more relevant for sensitivity.
From lesson: Hyperparameter Tuning — GridSearch, RandomSearch, Optuna
Why is joblib generally preferred over pickle when saving trained machine learning models with large weight matrices?
Answer: It uses memory mapping to handle large NumPy arrays more efficiently.. Joblib is specifically optimized for large NumPy arrays using memory mapping, which is faster and more memory-efficient than pickle. Pickle is not safer, does not inherently compress better, and is not cross-language compatible.
From lesson: Model Serialization — pickle and joblib
What happens if you use 'pickle.load()' on a file that was modified by a malicious actor?
Answer: It can execute arbitrary code contained within the file.. Pickle is fundamentally insecure because the unpickling process allows the execution of arbitrary code defined in the file. It does not error out, return wrong predictions, or ignore the payload; it simply runs what is inside.
From lesson: Model Serialization — pickle and joblib
When serializing a model pipeline, why is it best practice to include the scaler or encoder in the saved file?
Answer: To ensure that new input data is transformed using the exact parameters learned during training.. Models require the same transformation parameters (like mean/std for scaling) to function correctly. Including them ensures consistency. It does not cause automatic re-training, reduce file size, or prevent system crashes.
From lesson: Model Serialization — pickle and joblib
If you need to share a model across different environments or different versions of machine learning libraries, what is the main drawback of using pickle?
Answer: It is highly sensitive to the underlying software environment and library versions.. Pickle captures the internal state of Python objects, making it extremely dependent on library versions. It is not human-readable, it supports complex models, and the file size is not the primary issue.
From lesson: Model Serialization — pickle and joblib
Which of the following scenarios is the most appropriate use case for using joblib?
Answer: Storing a trained Random Forest with thousands of large decision trees.. Joblib's memory mapping makes it excellent for large models like Random Forests. It is not for web APIs (JSON is better), not for cross-language compatibility, and it offers no encryption for security.
From lesson: Model Serialization — pickle and joblib
When is it most appropriate to use a constant value (e.g., -1) to fill in missing data instead of statistical imputation?
Answer: When the fact that data is missing is itself a strong indicator of the target variable.. If the absence of data contains predictive signal, a specific indicator value allows the model to learn that missingness is a feature. Option 1 ignores the signal. Option 2 is better handled by mean/median. Option 4 is violated by constant imputation, which reduces variance.
From lesson: Handling Missing Data in ML
Why is 'Mean Imputation' generally discouraged for features with significant outliers?
Answer: Because the mean is highly sensitive to outliers, skewing the filled values away from the central tendency of the majority of data points.. The mean is affected by extreme values, so replacing missing entries with a mean influenced by outliers makes the imputed values unrepresentative of typical samples. The median is robust to outliers, making it a safer choice. The other options are either false or irrelevant.
From lesson: Handling Missing Data in ML
If you are training a model that explicitly handles missing values (such as certain tree-based algorithms), what is the main benefit of avoiding imputation?
Answer: It preserves the original data distribution and allows the model to find optimal splits for missing values.. Some algorithms learn the direction of missing values as a split point, preserving information without manual bias. Speed (Option 1) is often negligible. Option 3 is incorrect as the space is the same. Option 4 is never a guarantee.
From lesson: Handling Missing Data in ML
A dataset contains 50% missing values in a critical feature. What is the most robust initial approach?
Answer: Create a boolean 'is_missing' flag and then impute the numeric value.. Creating a flag preserves the information that the data was missing, while imputation fills the gap for the model to use. Dropping the feature (1) or rows (4) loses potentially useful information. Mode (2) is inappropriate for numeric features and risks high bias.
From lesson: Handling Missing Data in ML
What is the primary danger of using K-Nearest Neighbors (KNN) for imputation?
Answer: All of the above.. KNN requires scaling because distance metrics are sensitive to feature magnitude; it is memory-intensive for large datasets; and calculating neighbors for every missing point is far slower than mean/median imputation. Thus, all provided options are correct.
From lesson: Handling Missing Data in ML
Which statement best describes why we use MinMaxScaler instead of StandardScaler for certain neural network activations?
Answer: MinMaxScaler constrains features to a [0, 1] range, which aligns with activation functions like Sigmoid that saturate outside this range.. Option 2 is correct because the Sigmoid function maps inputs to a [0, 1] range, and having inputs in that same range prevents early saturation. Option 1 is false (that's StandardScaler). Option 3 is false (scaling doesn't change distribution shape). Option 4 is false (both require simple linear calculations).
From lesson: Feature Scaling — StandardScaler, MinMaxScaler
When applying StandardScaler, what happens to a feature that has a constant value across all observations?
Answer: The feature results in a division by zero error or NaN values.. Option 3 is correct because the formula involves subtracting the mean and dividing by the standard deviation. A constant feature has a standard deviation of zero, leading to division by zero. Option 1, 2, and 4 are incorrect because they don't describe the mathematical failure mode.
From lesson: Feature Scaling — StandardScaler, MinMaxScaler
You have a feature representing 'Income' with extreme outliers (billionaires). Which scaling choice is theoretically most robust?
Answer: Neither; both are sensitive to outliers as they rely on min/max or mean/std which are not robust to extreme values.. Option 3 is correct because both methods use global statistics (min/max or mean/std) that are heavily distorted by outliers. Option 1 and 4 are wrong because MinMaxScaler is the most sensitive to outliers. Option 2 is wrong because the mean is a non-robust statistic.
From lesson: Feature Scaling — StandardScaler, MinMaxScaler
If you are using a K-Nearest Neighbors (KNN) model, why is feature scaling mandatory?
Answer: KNN uses distance metrics like Euclidean distance, where larger magnitude features dominate the distance calculation.. Option 1 is correct because KNN relies on distance; a feature with a large range (e.g., salary) would overwhelm a feature with a small range (e.g., age). Option 2 is false (KNN is non-parametric). Option 3 is false (scaling does not directly regularize). Option 4 is false (KNN is not linear).
From lesson: Feature Scaling — StandardScaler, MinMaxScaler
After scaling your training data using StandardScaler, you find your model performs well. How should you transform the test data?
Answer: Use the mean and standard deviation derived from the training set to transform the test set.. Option 2 is correct because the test set must be transformed using the exact parameters learned from the training set to prevent data leakage. Option 1 is wrong (it would change the reference frame). Option 3 is wrong (it leaks information). Option 4 is wrong (the model expects the same feature scales used during training).
From lesson: Feature Scaling — StandardScaler, MinMaxScaler
What is the primary computational benefit of using Stochastic Gradient Descent over Batch Gradient Descent for a dataset with millions of records?
Answer: It requires significantly less memory as it only processes one sample at a time. Option 1 is wrong because SGD is stochastic and noisy. Option 2 is correct because the gradient is calculated per sample, preventing the need to store massive gradients in memory. Option 3 is wrong because feature scaling is still essential. Option 4 is wrong because SGD's path is famously oscillatory.
From lesson: Stochastic Gradient Descent
When training a model with SGD, why is it necessary to decay the learning rate over time?
Answer: To allow the model to converge by narrowing the search space near a minimum. Option 1 is wrong because decay happens later. Option 2 is a secondary benefit, not the purpose. Option 3 is correct because a high learning rate causes the model to jump over the minimum, whereas a lower rate allows it to settle. Option 4 is irrelevant to mathematical convergence.
From lesson: Stochastic Gradient Descent
If your loss function values are oscillating wildly during SGD training, which of the following is the most effective initial intervention?
Answer: Decreasing the learning rate or introducing momentum. Option 1 would increase oscillations. Option 2 does not address the step size. Option 3 is correct because smaller steps or momentum (which averages gradients) stabilize the updates. Option 4 would technically turn it into a mini-batch approach, which changes the definition of pure SGD.
From lesson: Stochastic Gradient Descent
What is the consequence of not shuffling your training data before every epoch in SGD?
Answer: The model may learn patterns based on the order of data rather than the features. Option 1 is wrong because it introduces bias. Option 2 is correct; data order can inadvertently bias the gradient updates. Option 3 is wrong because SGD doesn't change the underlying loss geometry. Option 4 is wrong because SGD is by definition an approximation.
From lesson: Stochastic Gradient Descent
In the context of SGD, what role does a 'momentum' term play in the parameter update rule?
Answer: It accumulates past gradients to accelerate progress in relevant directions and dampen oscillations. Option 1 is wrong because it uses all gradients. Option 2 is correct; momentum acts like a ball rolling down a hill, accumulating velocity. Option 3 is incorrect as momentum is additive to the update. Option 4 is incorrect as momentum helps escape, not terminate.
From lesson: Stochastic Gradient Descent
What is the primary purpose of using a learning rate warmup phase at the start of training?
Answer: To prevent gradient explosion in early iterations when weights are random.. Warmup stabilizes training when gradients are volatile due to random initialization. Option 0 is false because memorization is overfitting; option 2 is nonsensical; option 3 is incorrect as initialization is still necessary.
From lesson: Learning Rate and Schedulers
If your training loss is fluctuating wildly, what is the most appropriate first adjustment?
Answer: Reduce the learning rate or implement a decay schedule.. Wild fluctuations indicate the step size is too large to settle into the minimum. Option 0 worsens the issue; option 1 is irrelevant; option 3 adds unnecessary complexity without addressing the step size.
From lesson: Learning Rate and Schedulers
Why is the Cosine Annealing scheduler generally preferred over simple step-based decay?
Answer: It provides a smooth, gradual reduction that prevents sudden shocks to the model.. Cosine annealing provides a smooth curve that mimics the geometry of the loss landscape better than abrupt steps. Option 0 is false; option 2 is not the role of a scheduler; option 3 contradicts the definition of a scheduler.
From lesson: Learning Rate and Schedulers
In the context of 'ReduceLROnPlateau', why should you monitor validation loss rather than training loss?
Answer: Because validation loss indicates how well the model generalizes to unseen data.. We care about generalization. Reducing the rate based on training loss might just lead to overfitting. Option 0 is not a universal truth; option 2 is false regarding computation; option 3 is factually incorrect.
From lesson: Learning Rate and Schedulers
How does increasing the batch size interact with the optimal learning rate?
Answer: You should increase the learning rate because the gradient estimate is more accurate.. Larger batches provide a more reliable gradient direction, allowing for larger steps. Option 0 is backwards; option 1 ignores the statistical reality of batches; option 3 is arbitrary.
From lesson: Learning Rate and Schedulers
When comparing the weight decay behavior of L1 and L2, what happens to the coefficients as the penalty strength increases?
Answer: L1 creates sparse models by setting many coefficients to zero, while L2 shrinks coefficients toward zero but rarely hits zero.. L1 uses the absolute value of coefficients, creating a 'diamond' constraint region that hits axes at zero, enabling sparsity. L2 uses the square, creating a 'circular' region that shrinks values asymptotically but doesn't reach zero. The other options misstate the fundamental geometry of the penalty terms.
From lesson: Regularization — L1 (Lasso) and L2 (Ridge)
Why is feature scaling essential before applying L2 (Ridge) regularization?
Answer: The penalty term depends on the magnitude of the weights, which are biased by the scale of the corresponding input features.. L2 penalizes the sum of squared weights. If one feature is measured in thousands and another in decimals, the model will shrink the weight of the former more aggressively regardless of predictive power. Scaling ensures all features contribute equally to the penalty. Other options are technically incorrect regarding matrix inversion or the scope of regularization.
From lesson: Regularization — L1 (Lasso) and L2 (Ridge)
In a dataset with high multicollinearity among independent variables, how does Ridge regression behave compared to Ordinary Least Squares (OLS)?
Answer: Ridge regression remains stable by penalizing the squared magnitudes of coefficients, effectively distributing the weight across correlated features.. OLS becomes unstable with multicollinearity because small changes in data cause large swings in coefficients. Ridge regularization adds a small bias to reduce variance, distributing weights among correlated features rather than causing explosive coefficient values. Ridge is not faster, nor does it inherently drop correlated features.
From lesson: Regularization — L1 (Lasso) and L2 (Ridge)
What is the primary motivation for using the Elastic Net penalty over pure Lasso (L1)?
Answer: When features are highly correlated, Lasso tends to pick one randomly, whereas Elastic Net retains groups of correlated features.. Lasso's selection can be unstable with correlated features. Elastic Net adds an L2 component that 'groups' correlated features, keeping them in the model together. The other options are incorrect: Elastic Net is usually slower, Lasso does produce non-zero coefficients, and both remain linear models.
From lesson: Regularization — L1 (Lasso) and L2 (Ridge)
Suppose you are tuning the alpha parameter for Ridge regression. As alpha approaches infinity, what happens to the model?
Answer: The coefficients move toward zero, causing the model to underfit.. As alpha increases, the penalty becomes so dominant that the cost function is minimized only when all coefficients (except potentially the intercept) are zero. This leads to high bias and underfitting. OLS occurs at alpha=0, and increasing complexity is the opposite effect of high regularization.
From lesson: Regularization — L1 (Lasso) and L2 (Ridge)
What is the primary indicator that early stopping should trigger during the training of a neural network?
Answer: The validation error consistently increases over a predefined number of iterations.. The correct choice is the second one; early stopping monitors the validation set, and sustained increases in error signal overfitting. Option 1 is incorrect because zero training loss usually implies extreme overfitting. Option 3 relates to optimization convergence, not generalization. Option 4 relates to learning rate schedules, not stopping criteria.
From lesson: Early Stopping
Why is it important to use a 'patience' parameter when implementing early stopping?
Answer: To prevent stopping prematurely due to stochastic noise in mini-batch gradient descent.. Patience provides a buffer against temporary spikes in validation loss caused by the noisy nature of mini-batches. Option 1 is wrong because perfect learning is not the goal. Option 2 describes momentum or learning rate strategies. Option 4 is incorrect because validation is performed periodically, not every step.
From lesson: Early Stopping
How does early stopping technically act as a form of regularization?
Answer: It restricts the optimization process to a subspace of the parameter space that favors simpler models.. Early stopping restricts the training duration, effectively preventing the weights from reaching the high-complexity values that typically occur later in optimization. Option 1 describes L2 regularization. Option 3 describes data augmentation. Option 4 describes feature selection or pruning.
From lesson: Early Stopping
If you are using k-fold cross-validation, how should early stopping be handled?
Answer: Use the validation fold performance within each split to determine the stopping point independently for each model.. Each fold represents a different data split, so the optimal stopping epoch may differ; independent monitoring ensures each model is optimized correctly. Option 1 is flawed because different splits have different convergence rates. Options 3 and 4 disregard the purpose of cross-validation and monitoring generalization.
From lesson: Early Stopping
What is a potential downside of relying solely on early stopping for a model that is significantly too large for the task?
Answer: It may not be able to prevent the model from learning noise in the early stages of training.. If the model has high capacity, it might learn noise very quickly, meaning even early stopping might not prevent overfitting. Option 1 is correct in this context. Option 2 is irrelevant. Option 3 is false, as the architecture remains the same. Option 4 is not the primary issue, as reaching the global minimum is often undesirable in overparameterized models.
From lesson: Early Stopping
Why does a Support Vector Machine with a linear kernel require feature scaling, while a Decision Tree generally does not?
Answer: The SVM objective function relies on distance metrics between data points, making it sensitive to feature magnitude, whereas Decision Trees perform splits based on individual feature thresholds regardless of scale.. SVMs find a hyperplane that maximizes the margin between classes based on distance; large scales skew this margin. Decision trees use partitioning based on relative rankings or comparisons of a single feature, so the absolute scale is irrelevant. Options 1, 2, and 3 are conceptually incorrect regarding how these algorithms function.
From lesson: ML Interview Questions — Algorithms
In the context of K-Means clustering, what is the primary limitation of using Euclidean distance when features have significantly different variances?
Answer: Features with higher variance will disproportionately influence the cluster assignments, effectively masking the contribution of other features.. K-Means seeks to minimize the sum of squared distances to cluster centers. If one feature has a variance of 1000 and another of 0.1, the distance calculation is dominated by the former. Option 0 is false as convergence is guaranteed; Option 2 is false as K-Means assumes spherical clusters; Option 3 is a hardware-specific concern, not an algorithmic limitation.
From lesson: ML Interview Questions — Algorithms
Consider a Gradient Boosting model that is performing perfectly on the training set but poorly on the validation set. Which action is most appropriate to improve generalization?
Answer: Apply regularization constraints like subsampling (stochastic gradient boosting) or reducing the maximum depth of individual trees.. The symptoms indicate overfitting. Regularization (subsampling or depth control) limits the complexity of the trees. Increasing the learning rate or number of trees (Options 0 and 1) would exacerbate overfitting. Changing the loss function (Option 3) changes the objective but doesn't inherently prevent the model from over-fitting to the training noise.
From lesson: ML Interview Questions — Algorithms
Why is the 'curse of dimensionality' a critical concern for algorithms relying on distance calculations?
Answer: In high-dimensional spaces, the ratio of the distance to the nearest neighbor versus the farthest neighbor tends to approach one, making distance-based discrimination difficult.. In high-dimensional space, the distance between any two points becomes very similar (the distance to the nearest neighbor is almost as large as the distance to the farthest), rendering metrics like Euclidean distance uninformative. Option 0 is incorrect because complexity is usually linear with dimensions; 2 and 3 are not accurate descriptions of high-dimensional geometry.
From lesson: ML Interview Questions — Algorithms
When training a Logistic Regression model, why is the inclusion of an L2 penalty (Ridge regularization) preferred when features are highly correlated?
Answer: It prevents the coefficients from exploding in value, ensuring the model remains stable despite multicollinearity.. L2 regularization shrinks coefficients, preventing any single feature from dominating due to collinearity. L1 (Lasso) is the one that sets coefficients to zero (Option 0). Logistic regression loss is already convex (Option 2). L2 does not transform features into principal components (Option 3).
From lesson: ML Interview Questions — Algorithms
If your model performs significantly better on training data than on validation data, what is the most likely mathematical implication regarding the model's complexity?
Answer: The model has high variance, capturing noise as signal.. High variance means the model is overly complex and fits the training noise. Option 0 describes underfitting (high bias). Option 2 relates to optimization speed, not generalization. Option 3 is irrelevant to the training-validation gap.
From lesson: ML Interview Questions — Math and Stats
Why is the L2 regularization (Ridge) term mathematically preferred over the L1 regularization (Lasso) term when you suspect all features contribute small amounts to the target?
Answer: L2 penalty distributes the coefficient weights across all features.. L2 penalizes the square of coefficients, encouraging small but non-zero values, which is ideal for dense feature contributions. L1 (Lasso) drives coefficients to exactly zero, which is better for feature selection, not dense distributed contributions.
From lesson: ML Interview Questions — Math and Stats
In the context of Gradient Descent, what happens to the loss function if the learning rate is set too high?
Answer: The weights may overshoot the minimum and cause divergence.. A high learning rate causes large steps that can skip over the local minimum, leading to divergence. Option 0 and 1 are incorrect because high rates prevent stable convergence. Option 3 describes a method change, not an effect of the learning rate.
From lesson: ML Interview Questions — Math and Stats
How does increasing the number of predictors in a linear regression model affect the variance and bias of the model?
Answer: Bias decreases, and variance increases.. More predictors allow the model to fit the data more closely (lower bias), but also increase the sensitivity to fluctuations in the data (higher variance). The other options fail to account for the inverse relationship between bias and variance in this context.
From lesson: ML Interview Questions — Math and Stats
When evaluating a classifier on a highly imbalanced dataset, why is the F1-score generally more informative than standard accuracy?
Answer: F1-score accounts for both precision and recall, whereas accuracy can be misleadingly high if the majority class dominates.. Accuracy is misleading because a model can predict the majority class 100% of the time and still have high accuracy. F1-score uses the harmonic mean of precision and recall to provide a balanced metric. Option 0 is false, Option 2 is mathematically wrong, and Option 3 misinterprets the purpose of the metric.
From lesson: ML Interview Questions — Math and Stats
When designing a recommendation system, why might you prioritize a two-stage architecture (retrieval followed by ranking) over a single-stage model?
Answer: To balance computational latency with the ability to score a large candidate corpus.. Retrieval filters millions of items to hundreds, which is necessary for latency requirements in real-time systems, whereas scoring millions of items with a complex ranker is too slow. Option 0 is false as ranking models handle complex features. Option 2 is false as the architecture choice doesn't solve data sparsity. Option 3 is irrelevant.
From lesson: ML System Design Questions
Which of the following best describes the benefit of implementing an online evaluation strategy (A/B testing) compared to offline metrics?
Answer: It measures the actual causal impact of the model on user behavior and business KPIs.. Offline metrics are proxies, but online A/B testing measures real-world impact on user behavior. Option 0 is wrong because skew analysis is still required for robustness. Option 1 is wrong because A/B tests are typically slower than offline validation. Option 3 is wrong because A/B tests require high traffic for significance.
From lesson: ML System Design Questions
What is the primary risk of using a 'push' rather than 'pull' approach for serving features in a low-latency ML system?
Answer: Pulling features directly from a database often introduces unacceptable latency during request time.. Pulling features from a database during a request adds network overhead (latency). Pushing (pre-computing and caching) is faster. Option 0 is not the primary risk. Option 2 is incorrect. Option 3 is incorrect as both can support transformations.
From lesson: ML System Design Questions
In the context of feedback loops in ML systems, how does 'bias amplification' occur?
Answer: When the model's predictions influence the future data collected, causing the model to overfit to its own previous outputs.. Feedback loops occur when model outputs drive user behavior, which is then captured as new training data, reinforcing the model's biases. Options 1, 2, and 3 describe other issues like overfitting, convergence, or sampling errors, not feedback loops.
From lesson: ML System Design Questions
If you observe a significant drop in model performance over time without a corresponding change in the training pipeline, what is the most likely cause?
Answer: Data or concept drift where the distribution of input data or the relationship between features and labels has shifted.. Drift is the most common reason for 'silent' failures in production ML. Leakage (Option 0) usually causes inflated metrics immediately. Options 1 and 3 are development-stage errors, not time-based degradation.
From lesson: ML System Design Questions
