A two-stage machine learning system for industrial predictive maintenance, built on the AI4I 2020 Predictive Maintenance Dataset (10,000 synthetic sensor readings from a milling machine simulator, UCI/Kaggle).
Live demo: https://predictive-maintenance-363846849117.us-central1.run.app
A maintenance engineer doesn't just want "will this machine fail" — they want to know how urgent and what to check first. This project builds that as two chained models instead of one flat classifier:
- Stage 1 — Failure risk (binary). Given live sensor readings, what's the probability this machine fails? The positive class is only 3.5% of the data, so this is a genuine imbalanced-classification problem, not a toy one — raw accuracy is meaningless here (predicting "no failure" always gets 96.5%).
- Stage 2 — Root cause (multiclass). Conditional on elevated risk, which of 5 failure modes is most likely (heat dissipation, power, overstrain, tool wear, or a random failure)? This is the more useful question in practice — it tells the engineer what to inspect.
- Source: AI4I 2020 Predictive Maintenance Dataset (10,000 rows, 6 raw
sensor/process columns + a product quality tier
Type∈ {L, M, H}). - Known label inconsistency, found and fixed: 27 rows have a
Targetflag that disagrees with the hand-labeledFailure Type(9 rows marked as a failure withFailure Type = "No Failure"; 18 rows marked as no failure but assigned a real failure type).Failure Typewas treated as authoritative andTargetwas re-derived from it (src/data_prep.py). The mismatched rows are saved toreports/label_inconsistencies.csv. - Feature engineering: Kelvin → Celsius, a temperature-differential feature (process − air temp, a known driver of heat-dissipation failures), and a mechanical power feature (torque × angular velocity, a known driver of power failures) — both directly reflect the physical failure mechanisms in this dataset.
Both stages compared Logistic Regression, Random Forest, XGBoost, and LightGBM.
Stage 1 (binary, 80/20 stratified split, class-weighted vs. SMOTE):
| Model | Precision | Recall | F1 | ROC-AUC | PR-AUC |
|---|---|---|---|---|---|
| XGBoost (class-weighted) | 0.92 | 0.77 | 0.84 | 0.985 | 0.909 |
| LightGBM (class-weighted) | 0.93 | 0.74 | 0.83 | 0.986 | 0.905 |
| Random Forest (class-weighted) | 0.90 | 0.79 | 0.84 | 0.983 | 0.899 |
| Logistic Regression (class-weighted) | 0.18 | 0.90 | 0.29 | 0.956 | 0.536 |
PR-AUC (not accuracy or even ROC-AUC) is the model-selection metric, since
it's the honest metric under severe class imbalance. XGBoost wins and
catches 77% of real failures while keeping false alarms low (92%
precision). Full comparison: reports/binary_model_comparison.csv.
Stage 2 (multiclass, 5-fold stratified cross-validation — a single 80/20 split would be too noisy with only 348 failure rows and one class as small as 18 examples):
| Model | CV macro-F1 |
|---|---|
| Random Forest | 0.948 |
| XGBoost | 0.927 |
| LightGBM | 0.921 |
| Logistic Regression | 0.840 |
Random Forest wins, with macro-F1 0.95 across all 5 failure modes
including the 18-example "Random Failures" class (F1 0.91 on that class
alone). Full report: reports/multiclass_best_model_report.txt.
SHAP feature-importance plots for both models are in reports/. The
binary model's top drivers — tool wear, mechanical power, and rotational
speed — line up with the physical failure mechanisms in the dataset
(reports/shap_binary_bar.png, reports/shap_binary_beeswarm.png,
reports/shap_multiclass_bar.png). The demo app also shows a live
per-prediction SHAP breakdown.
predictive_maintenance/
├── predictive_maintenance.csv raw source data
├── src/
│ ├── data_prep.py clean data, fix label inconsistency, feature engineering
│ ├── utils.py shared feature spec
│ ├── train_binary.py stage-1 model comparison + training
│ ├── train_multiclass.py stage-2 model comparison + training
│ └── explain.py SHAP plot generation
├── models/ trained models (joblib) + metadata
├── reports/ metrics, classification reports, SHAP plots
├── app/app.py Streamlit inference demo
├── Dockerfile.dev dev/training image (pandas/sklearn/xgboost/lightgbm/shap/streamlit)
└── predictive_maintenance_EDA.ipynb original exploratory analysis
Requires Docker (used here because this machine's Windows Application
Control policy blocks freshly pip-installed native DLLs in local venvs —
see Dockerfile.dev).
docker build -f Dockerfile.dev -t predmaint-dev .
docker run --rm -v "$PWD":/app -w /app predmaint-dev python src/data_prep.py
docker run --rm -v "$PWD":/app -w /app predmaint-dev python src/train_binary.py
docker run --rm -v "$PWD":/app -w /app predmaint-dev python src/train_multiclass.py
docker run --rm -v "$PWD":/app -w /app predmaint-dev python src/explain.py
docker run --rm -p 8501:8501 -v "$PWD":/app -w /app predmaint-dev streamlit run app/app.py