A Half-Day Return to Machine Learning with Fraud Detection
A short fraud-detection project helped me revisit class imbalance, Random Forest, SMOTETomek, and precision-recall metrics.
After a few years away from traditional machine learning, a small fraud-detection project was a good way to rebuild my instincts.
The recent focus on neural networks and large language models made it easy to forget how useful classical machine learning still is.
I returned to the basics with a credit-card fraud dataset from Kaggle. The project was small, but it brought back several important lessons about imbalance, evaluation, and the limits of a quick experiment.
The dataset
The dataset contains 284,315 transactions and 492 fraud cases. Fraud represents only 0.172% of the records, so a model can achieve high accuracy by predicting “not fraud” almost every time.
Most transaction features are anonymized PCA components such as V1, V2, and V14. The original merchant and location information is hidden, leaving Time and Amount as the most interpretable fields.
That makes feature engineering simpler, but it also limits how much business context I can recover from the data.
A Random Forest baseline
I used a Random Forest Classifier as the first model. It combines many decision trees and can capture non-linear relationships without requiring a complex neural-network setup.
It also provides feature-importance estimates, which gave me a way to inspect which anonymized variables influenced the predictions.
I trained the model with SMOTETomek and did not compare it with a larger set of algorithms. I only spent half a day on the exercise, so this is a learning baseline rather than a production fraud model.
Handling class imbalance
There are two broad ways to address imbalance: increase the number of minority-class examples or reduce the number of majority-class examples.
Downsampling discards legitimate transactions. Simple duplication can overfit the same fraud cases.
I chose SMOTETomek. It combines synthetic oversampling with the removal of some overlapping samples.
The goal is not to manufacture a perfect dataset. It is to give the model more useful minority-class examples while reducing some noisy overlap.
Why accuracy is not enough
Accuracy hides the problem here. A model that labels every transaction as legitimate can still look impressive because fraud is so rare.
I therefore focused on the Area Under the Precision-Recall Curve (AUPRC). Precision measures how many flagged transactions are actually fraud, while recall measures how many fraud cases the model catches.
AUPRC gives the rare positive class more attention than accuracy does. It is a more useful lens for this dataset, though a real system would still need threshold selection, calibration, monitoring, and careful validation.
What I learned
The half-day experiment was enough to rebuild my confidence with traditional machine learning. The most valuable part was not the Random Forest itself; it was remembering to ask whether the metric matches the problem.
Fraud detection is a high-stakes domain, so this notebook is not evidence of a production-ready detector. It is a compact refresher on why data distribution and evaluation design matter.