Classic machine learning

Logistic regression

Like turning a risk score into a chance of rain, logistic regression maps a weighted score to a probability. For two classes, a sigmoid keeps the output between zero and one. A chosen threshold turns that probability into a class decision.

-4-3-2-10123400.20.40.60.81Weighted score zProbability of class 1Score 0 gives 0.5
Sigmoid
Scores -2, 0 and 2 map to probabilities about 0.12, 0.50 and 0.88.

How it works

  1. Compute a score $z=b+w^Tx$ from each example.
  2. Apply the sigmoid to get the probability of class 1.
  3. Fit weights by minimizing binary log loss, optionally with a weight penalty.
  4. Evaluate probabilities and choose a decision threshold for the costs of mistakes.

$$P(y=1\mid x)=\frac{1}{1+e^{-(b+w^Tx)}}$$

The knobs that matter

Regularization strength
More shrinkage reduces coefficient size and can reduce overfitting.
Class weights
Change how much each class contributes to the loss.
Decision threshold
Trades precision against recall without refitting weights.

What interviewers ask

  • Why use log loss rather than squared error here?
  • Can logistic regression learn XOR from the two raw inputs?
  • How do you interpret a coefficient in terms of odds?
  • How would you evaluate it on an imbalanced dataset?

Common mistakes

  • Assuming probability is linear in the features rather than the log odds.
  • Using accuracy alone or assuming 0.5 is always the right threshold.

When to use it

Use it for a fast, interpretable classification baseline, including sparse features. Raw linear scores need feature engineering to capture interactions such as XOR.

Implement it (4 problems)

Build it step by step

Learn it

Interview questions about logistic regression