Classic machine learning
Logistic regression
Like turning a risk score into a chance of rain, logistic regression maps a weighted score to a probability. For two classes, a sigmoid keeps the output between zero and one. A chosen threshold turns that probability into a class decision.
Sigmoid
How it works
- Compute a score $z=b+w^Tx$ from each example.
- Apply the sigmoid to get the probability of class 1.
- Fit weights by minimizing binary log loss, optionally with a weight penalty.
- Evaluate probabilities and choose a decision threshold for the costs of mistakes.
$$P(y=1\mid x)=\frac{1}{1+e^{-(b+w^Tx)}}$$
The knobs that matter
- Regularization strength
- More shrinkage reduces coefficient size and can reduce overfitting.
- Class weights
- Change how much each class contributes to the loss.
- Decision threshold
- Trades precision against recall without refitting weights.
What interviewers ask
- Why use log loss rather than squared error here?
- Can logistic regression learn XOR from the two raw inputs?
- How do you interpret a coefficient in terms of odds?
- How would you evaluate it on an imbalanced dataset?
Common mistakes
- Assuming probability is linear in the features rather than the log odds.
- Using accuracy alone or assuming 0.5 is always the right threshold.
When to use it
Use it for a fast, interpretable classification baseline, including sparse features. Raw linear scores need feature engineering to capture interactions such as XOR.
Implement it (4 problems)
Build it step by step
- NumPy Text Classifier from Scratch27 steps, your code runs in the browser