Machine Learning Exam Preparation: Key Concepts and Formulas

Posted on Aug 9, 2026 in Mathematics and Computer Science

MACHINE LEARNING — EXAM CHEAT SHEET

Built from your notes’ Model Question Paper — the exact questions likely to repeat

MODULE I — ML Foundations & Concept Learning

Q1(a). What is Machine Learning? Applications? [4M]

• Definition: A field where computers learn patterns from data or experience instead of being explicitly programmed for every rule.
• Mitchell’s formal definition: A program learns from experience E with respect to task T and performance measure P, if performance on T (measured by P) improves with E.
• Applications: Spam filtering, speech/image recognition, recommendation systems, fraud detection, self-driving cars, and data mining for hidden patterns.

Q1(b). Four Main Challenges of Machine Learning [8M]

• Insufficient training data: Most algorithms need thousands to millions of examples to work well.
• Non-representative training data: Sampling noise (small sets) or sampling bias (flawed collection method) causes poor generalization.
• Poor-quality data: Errors, outliers, and noise hide the real pattern; cleaning data consumes most of a data scientist’s time.
• Irrelevant features: Feature engineering (selection, extraction, and creating new features) is critical; garbage in, garbage out.
• Bonus (often asked together): Overfitting — model is too complex and memorizes noise (fix via simpler model, more data, or regularization). Underfitting — model is too simple (fix via more complex model, better features, or less regularization).

Q1(c) / Q9(a). Candidate Elimination Algorithm [8–10M]

• Maintains two boundary sets over the hypothesis space consistent with all examples seen so far: S (specific boundary) and G (general boundary).
• Initialize: S = most specific hypothesis (all ∅), G = most general hypothesis (all ‘?’).
• Positive example: Generalize S minimally so it covers the example; remove from G any hypothesis inconsistent with it.
• Negative example: Specialize G minimally to exclude the example; remove from S any hypothesis that wrongly covers it.
• Version space: All hypotheses between S and G that are consistent with the data — practice the ‘Japanese Economy Car’ style table trace.

Q2(a). Find-S Algorithm [10M]

• Goal: Find the maximally specific hypothesis consistent with the positive examples only (ignores negatives).
• Step 1: h = <∅, ∅, …, ∅> (most specific).
• Step 2: For each positive example — for each attribute, if the example’s value equals h’s value, keep it; otherwise, replace it with ‘?’.
• Step 3: Ignore every negative example completely.
• Step 4: Final h is the output. Practice with the EnjoySport dataset (Sky, AirTemp, Humidity, Wind, Water, Forecast).
• Limitations: Cannot tell if it converged to the only correct hypothesis, cannot handle noisy/inconsistent data, and ignores negative examples entirely.

Q2(b) & Q2(c). Unbiased Learner / Task-Performance-Experience [6M + 4M]

• Unbiased learner: Makes no prior assumptions about the target concept (hypothesis space = all possible concepts). However, this means it cannot generalize beyond seen examples (no inductive bias).
• Checkers learning problem: T: play checkers, P: % of games won, E: games played against itself.
• Robot driving problem: T: drive on a public 4-lane highway, P: average distance before human intervention, E: sequence of images and steering commands recorded while observing a human driver.

MODULE II — Data Preparation & Model Evaluation

Q3(a). Preparing Data for ML [10M]

• Data Cleaning: Handle missing values (drop rows/columns or impute with median/mean); remove outliers and duplicates.
• Handling text & categorical attributes: OrdinalEncoder for ordered categories; OneHotEncoder for nominal categories (avoids implying false order).
• Feature scaling: Min-Max scaling (normalization, scales to [0,1]) vs. Standardization (zero mean, unit variance). Standardization is less affected by outliers.

Q3(b). Grid Search vs. Randomized Search [10M]

Q4(a). Cross-Validation, Confusion Matrix, Precision & Recall [10M]

• Cross-validation (e.g., k-fold): Splits data into k folds, trains on k−1 and tests on the remaining fold, rotating through all folds. This reduces overfitting risk and gives a more reliable accuracy estimate.
• Confusion Matrix: Rows = actual class, columns = predicted class → TP, TN, FP, FN cells; the base for every other classification metric.

Precision = TP / (TP + FP) | Recall = TP / (TP + FN) | F1 = 2 · (Precision · Recall) / (Precision + Recall)

Q4(b). Multilabel vs. Multiclass vs. Multioutput Classification [10M]

• Multiclass: Each instance gets exactly one label from more than two possible classes (e.g., digit 0–9).
• Multilabel: Each instance can get several labels simultaneously (e.g., an image tagged both ‘cat’ and ‘outdoor’).
• Multioutput-multiclass: A generalization where each label itself can take more than two values (e.g., predicting multiple pixel values to remove noise from an image).

MODULE III — Optimization & Regularization

Q5(a). Gradient Descent & Its Types [10M]

• Gradient Descent: Iteratively adjusts model parameters in the direction that reduces the cost function fastest (negative gradient), scaled by the learning rate.
• Batch GD: Uses the entire training set per step — smooth convergence but slow on large data.
• Stochastic GD (SGD): Uses one random instance per step — fast, but follows a noisy path around the minimum; needs a decreasing learning rate to settle.
• Mini-batch GD: Uses a small random subset per step — a practical middle ground that benefits from hardware vectorization.

Q5(b). Regularized Linear Models — 3 Ways to Constrain Weights [10M]

• Ridge Regression (L2): Adds a λ · Σ(weight²) penalty to the cost function; shrinks weights toward zero but rarely to exactly zero.
• Lasso Regression (L1): Adds a λ · Σ|weight| penalty; can shrink weights all the way to zero, performing automatic feature selection.
• Elastic Net: Mixes L1 and L2 penalties (controlled by l1_ratio); useful when many features are correlated and you want Lasso’s sparsity with Ridge’s stability.

Note: Nonlinear SVM (Polynomial/RBF kernel) and Quadratic Programming (Q6 in the model paper) are not covered in depth in your uploaded notes — flag this with your faculty or reference Hands-On ML Ch. 5 if it is compulsory.

MODULE IV — Decision Trees & Ensemble Methods

Q7(a)/(b)/(c). Decision Trees & CART Algorithm [10M+6M+4M]

• A decision tree splits data at each node using the attribute/threshold that best reduces impurity, down to leaf nodes holding the predicted class/value.
• CART (Classification And Regression Trees): A greedy algorithm that searches every feature k and threshold t_k, picking the pair producing the purest split (weighted by subset size).
• Splitting criterion: Gini impurity for classification, residual/MSE reduction for regression.
• Pruning: Removes low-value nodes to fight overfitting (cost-complexity pruning, information-gain pruning).
• Regression trees: Predict a continuous average value per leaf; a key weakness is instability — small changes in data can produce a very different tree (high variance).

Q8(a). Bagging & Pasting, Voting Classifiers [10M]

• Bagging: The same algorithm trained on random subsets sampled WITH replacement (bootstrap) — reduces variance.
• Pasting: The same idea but sampling WITHOUT replacement.
• Aggregation: Both aggregate predictions by majority vote (classification) or averaging (regression); they train in parallel and scale well.
• Voting Classifier: Combines predictions from several different models. Hard voting = majority class wins; Soft voting = averages predicted probabilities (usually more accurate if classifiers are well-calibrated).

Q8(b). Boosting — AdaBoost & Gradient Boosting [10M]

• AdaBoost: Trains weak learners sequentially; each new learner focuses more on instances the previous ones misclassified (by increasing their weights); the final prediction is a weighted vote.
• Gradient Boosting: Each new learner is trained to predict the residual errors of the combined previous learners, gradually driving residuals toward zero.
• Histogram-based Gradient Boosting: Speeds this up by binning continuous features before building trees.

MODULE V — Bayesian Learning

Q9(a). Bayes Theorem & Link to Concept Learning [10M]

P(h|D) = [ P(D|h) · P(h) ] / P(D)

• P(h): Prior probability of hypothesis h; P(D): Prior probability of data D; P(D|h): Likelihood; P(h|D): Posterior probability of h given D.
• Link to concept learning: Choosing the MAP (Maximum a Posteriori) hypothesis — the h that maximizes P(h|D) — is equivalent to the FIND-S/Candidate-Elimination goal of finding the hypothesis best supported by the training data under a suitable prior.

Q9(b). Maximum Likelihood Hypothesis for Predicting Probabilities [10M]

• MLE: Picks the hypothesis parameters that make the observed training data most probable — maximizes P(D|h) rather than the posterior P(h|D).
• Under a uniform prior over hypotheses, the MAP hypothesis reduces to the ML hypothesis.
• For regression with Gaussian noise, maximizing likelihood is equivalent to minimizing squared error — this is why least-squares regression has a probabilistic justification.

Q10(a). Naïve Bayes Classifier — With Example [10M]

• Assumes all features are conditionally independent given the class (the ‘naïve’ assumption) — this makes computing P(class|features) tractable even with many features.

P(class|x1,…,xn) ∝ P(class) · Π P(xi|class)

• Steps: Compute prior P(class) from training data, compute each P(feature|class) from frequency counts, multiply together for each class, and pick the class with the highest product.
• Practice: Use the ‘Buys Computer’ age/income/student/credit-rating dataset style problem — walk through the exact multiplication for a new example.

Q10(b). EM (Expectation-Maximization) Algorithm — Derivation [10M]

• Used when data has latent (hidden/unobserved) variables — e.g., clustering with unknown group membership or missing data.
• Initialization step: Start with random or initial guesses for the model parameters.
• E-step (Expectation): Using current parameters, estimate the missing values or latent variable assignments (soft assignments — probabilities, not hard labels).
• M-step (Maximization): Re-estimate the parameters that maximize the expected likelihood given the estimates from the E-step.
• Convergence step: Repeat E and M steps until parameters stop changing significantly.
• Applications: Gaussian Mixture Models, k-means style clustering, and filling in missing data.

QUICK FORMULA REFERENCE SHEET

Tip: The model paper gives 10 questions × 20 marks. If you can answer all Q1–Q10 above confidently, you cover the entire expected paper.