Project idea · ungraded

Spam or Fraud Classifier

Build a classifier for a problem where the interesting class is rare — spam messages or fraudulent transactions — and choose a decision threshold that reflects real costs: a missed fraud is expensive, but so is blocking a real customer.

You will use

Milestones

Build the smallest working version first. Ticks are saved in this browser only.

01Load the data, measure the class imbalance and fit a majority-class baseline.
02Train a logistic regression pipeline and a gradient-boosting model with stratified cross-validation.
03Compare them with ROC AUC and average precision, and plot both curves.
04Choose a threshold from the precision–recall curve on validation data for a target recall.
05Report the confusion matrix and the cost per 1,000 cases at that threshold on the test set.

Stretch goals

Try class_weight='balanced' and compare it with moving the threshold.
For SMS spam, turn the messages into TF-IDF features (see the NLP topic).
Check how the precision changes if fraud becomes twice as rare after deployment.

Data

SMS Spam Collection (UCI): 5,574 labelled text messages, about 13% spam.
Synthetic transactions: sklearn.datasets.make_classification(n_samples=20000, weights=[0.98, 0.02], random_state=0) (documentation).

Run it yourself

pip install numpy pandas scikit-learn matplotlib
python fraud.py