Skip to content

About

Two-stage ML system: failure risk detection + root-cause classification on the AI4I 2020 predictive maintenance dataset

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Predictive Maintenance: Failure Risk & Root-Cause Classification

A two-stage machine learning system for industrial predictive maintenance, built on the AI4I 2020 Predictive Maintenance Dataset (10,000 synthetic sensor readings from a milling machine simulator, UCI/Kaggle).

Live demo: https://predictive-maintenance-363846849117.us-central1.run.app

Problem

A maintenance engineer doesn't just want "will this machine fail" — they want to know how urgent and what to check first. This project builds that as two chained models instead of one flat classifier:

  1. Stage 1 — Failure risk (binary). Given live sensor readings, what's the probability this machine fails? The positive class is only 3.5% of the data, so this is a genuine imbalanced-classification problem, not a toy one — raw accuracy is meaningless here (predicting "no failure" always gets 96.5%).
  2. Stage 2 — Root cause (multiclass). Conditional on elevated risk, which of 5 failure modes is most likely (heat dissipation, power, overstrain, tool wear, or a random failure)? This is the more useful question in practice — it tells the engineer what to inspect.

Data notes

  • Source: AI4I 2020 Predictive Maintenance Dataset (10,000 rows, 6 raw sensor/process columns + a product quality tier Type ∈ {L, M, H}).
  • Known label inconsistency, found and fixed: 27 rows have a Target flag that disagrees with the hand-labeled Failure Type (9 rows marked as a failure with Failure Type = "No Failure"; 18 rows marked as no failure but assigned a real failure type). Failure Type was treated as authoritative and Target was re-derived from it (src/data_prep.py). The mismatched rows are saved to reports/label_inconsistencies.csv.
  • Feature engineering: Kelvin → Celsius, a temperature-differential feature (process − air temp, a known driver of heat-dissipation failures), and a mechanical power feature (torque × angular velocity, a known driver of power failures) — both directly reflect the physical failure mechanisms in this dataset.

Modeling & results

Both stages compared Logistic Regression, Random Forest, XGBoost, and LightGBM.

Stage 1 (binary, 80/20 stratified split, class-weighted vs. SMOTE):

Model Precision Recall F1 ROC-AUC PR-AUC
XGBoost (class-weighted) 0.92 0.77 0.84 0.985 0.909
LightGBM (class-weighted) 0.93 0.74 0.83 0.986 0.905
Random Forest (class-weighted) 0.90 0.79 0.84 0.983 0.899
Logistic Regression (class-weighted) 0.18 0.90 0.29 0.956 0.536

PR-AUC (not accuracy or even ROC-AUC) is the model-selection metric, since it's the honest metric under severe class imbalance. XGBoost wins and catches 77% of real failures while keeping false alarms low (92% precision). Full comparison: reports/binary_model_comparison.csv.

Stage 2 (multiclass, 5-fold stratified cross-validation — a single 80/20 split would be too noisy with only 348 failure rows and one class as small as 18 examples):

Model CV macro-F1
Random Forest 0.948
XGBoost 0.927
LightGBM 0.921
Logistic Regression 0.840

Random Forest wins, with macro-F1 0.95 across all 5 failure modes including the 18-example "Random Failures" class (F1 0.91 on that class alone). Full report: reports/multiclass_best_model_report.txt.

Explainability

SHAP feature-importance plots for both models are in reports/. The binary model's top drivers — tool wear, mechanical power, and rotational speed — line up with the physical failure mechanisms in the dataset (reports/shap_binary_bar.png, reports/shap_binary_beeswarm.png, reports/shap_multiclass_bar.png). The demo app also shows a live per-prediction SHAP breakdown.

Project structure

predictive_maintenance/
├── predictive_maintenance.csv       raw source data
├── src/
│   ├── data_prep.py                 clean data, fix label inconsistency, feature engineering
│   ├── utils.py                     shared feature spec
│   ├── train_binary.py              stage-1 model comparison + training
│   ├── train_multiclass.py          stage-2 model comparison + training
│   └── explain.py                   SHAP plot generation
├── models/                          trained models (joblib) + metadata
├── reports/                         metrics, classification reports, SHAP plots
├── app/app.py                       Streamlit inference demo
├── Dockerfile.dev                   dev/training image (pandas/sklearn/xgboost/lightgbm/shap/streamlit)
└── predictive_maintenance_EDA.ipynb original exploratory analysis

Running it

Requires Docker (used here because this machine's Windows Application Control policy blocks freshly pip-installed native DLLs in local venvs — see Dockerfile.dev).

docker build -f Dockerfile.dev -t predmaint-dev .
docker run --rm -v "$PWD":/app -w /app predmaint-dev python src/data_prep.py
docker run --rm -v "$PWD":/app -w /app predmaint-dev python src/train_binary.py
docker run --rm -v "$PWD":/app -w /app predmaint-dev python src/train_multiclass.py
docker run --rm -v "$PWD":/app -w /app predmaint-dev python src/explain.py
docker run --rm -p 8501:8501 -v "$PWD":/app -w /app predmaint-dev streamlit run app/app.py

About

Two-stage ML system: failure risk detection + root-cause classification on the AI4I 2020 predictive maintenance dataset

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages